For this project you will take the role of a consultant hired by a real estate investment firm in Ames, Iowa, a mid-west town in the United States, to analyze data in order to help provide insight into how the firm should invest for highest profits, and to quantify and communicate to the company management what types of real estate properties are good investments and why. They have provided you with data on housing sales from between 2006 to 2010 that contains information about the characteristics of the house (number of bedrooms, number of bathrooms, square footage, etc.) and the house’s sale price. The codebook for this data set is available online here or in the Data folder in your repo.

About the Data Analysis Project

It’s generally a bad idea to buy the most expensive house in the neighborhood. And remember the real estate agents’ mantra: Location, location, location! Keep in mind that the goal is to make money for your investors, and hence investing in a property that is overvalued (costing more than it is worth) is rarely a good idea. This means that it’s critical to know which properties are overvalued and which are undervalued. The company that hired you has many questions for you about the housing market. It is up to you to decide what methods you want to use (frequentist or Bayesian) to answer these questions, and implement them to help to identify undervalued and overvalued properties.

You will have three data sets: a subset for training, a subset for testing, and a third subset for validation. You will be asked to do data exploration and build your model (or models) initially using only the training data. Then, you will test your model on the testing data, and finally validate using the validation data. We are challenging you to keep your analysis experience realistic, and in a realistic scenario you would not have access to all three of these data sets at once. You will be able to see on our scoreboard how well your team is doing based on its predictive performance on the testing data. After your project is turned in you will see the final score on the validation set.

All members of the team should contribute equally and answer any questions about the analysis at the final presentation.

For your analysis create a new notebook named “project.Rmd” and update accordingly rather than editing this.

Read in Training Data

To get started read in the training data.

library(dplyr)

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
library(tidyr)
load("ames_train.Rdata")
print(paste0("The dataset has ", dim(ames_train)[1], " number of observations and ", dim(ames_train)[2], " features"))
[1] "The dataset has 1500 number of observations and 81 features"
#Variables with NA's and their proportion of missing data
miss = apply(is.na(ames_train), 2, sum)
miss_prop = round(miss[miss>0]/nrow(ames_train), 3)
print(miss_prop) 
  Lot.Frontage          Alley   Mas.Vnr.Area      Bsmt.Qual      Bsmt.Cond  Bsmt.Exposure 
         0.188          0.930          0.004          0.034          0.034          0.034 
BsmtFin.Type.1 BsmtFin.Type.2 Bsmt.Full.Bath Bsmt.Half.Bath   Fireplace.Qu    Garage.Type 
         0.034          0.034          0.001          0.001          0.474          0.046 
 Garage.Yr.Blt  Garage.Finish    Garage.Qual    Garage.Cond        Pool.QC          Fence 
         0.047          0.046          0.047          0.047          0.995          0.789 
  Misc.Feature 
         0.965 
which(miss_prop>0.5) # four features have greater than 50% of data "missing" -- drop these variables
       Alley      Pool.QC        Fence Misc.Feature 
           2           17           18           19 

Notes about data cleaning:

We dropped “utilities” (type of utilities available) since in the training set, only 2 observations did not have all the utilities (electricity, gas, water and sewage). Intuitively, most modern property are equipped with these basic public utilities and keeping the variable would therefore be unnecessary.

We also dropped “condition 2” (proximity to various conditions if more than one is present) since it seemed redundant from our training set. Given Condition 1, only 12 properties were not close to normal conditions.

In the original scale, 1990 and 1900 would not be much different. Therefore, we changed the scale to the number of years since last construction or remodelling, subtracted from year 2010 (the end year in the dataset).

Another variable dropped in our model was “roof material” since only 1% of the property used material other than “standard composite shingle”. Similarly, “heating” was also dropped since more than 95% of the property has gas forced warm air furnace (GasA) instead of other types of heating.

For “exterior quality” and “exterior condition” , we recoded these ordinal variables to 1-5 to replace the original scale of conditions (from poor to excellent). Similarly, we recoded “basement exposure” and “basement rating” except that the new scale would start from 0 for properties without basement.

Exter Qual (Ordinal): Evaluates the quality of the material on the exterior

  Ex       Excellent
   Gd       Good
   TA       Average/Typical
   Fa       Fair
   Po       Poor

Exter Cond (Ordinal): Evaluates the present condition of the material on the exterior

  Ex       Excellent
   Gd       Good
   TA       Average/Typical
   Fa       Fair
   Po       Poor

Continuous variables such as “1st floor square feet” and “2nd floor square feet” were log-transformed for interpretation purpose.

For variable “functional”, we recoded different ordinal levels into binary levels — typical functionality or not, including minor and major deductions.

We summed up the number of bathrooms to one continuous variable. Note that one half-bathroom would be coded as 0.5.

# Did not remove any NA entries in Lot.frontage
data=ames_train
data <- data  %>%
  #filter(!is.na(Lot.Frontage)) %>% 
  mutate(MS.SubClass= factor(MS.SubClass)) %>%
  mutate(Alley =  factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
  mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add))  %>%
  filter(!is.na(Mas.Vnr.Area))  %>% 
  mutate(Bsmt.YN = 1*(!is.na(Bsmt.Qual))) %>%
  mutate(Bsmt.Qual =  factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Qual = relevel(Bsmt.Qual, ref="None")) %>%
  mutate(Bsmt.Cond =  factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Cond = relevel(Bsmt.Cond, ref="None")) %>%
  mutate(Bsmt.Exposure =  factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Exposure = relevel(Bsmt.Exposure, ref="None")) %>%
  mutate(BsmtFin.Type.1=  factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "None"), exclude = NULL)) %>%
  mutate(BsmtFin.Type.1 = relevel(BsmtFin.Type.1, ref="None")) %>%
  mutate(BsmtFin.Type.2=  factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "None"), exclude = NULL)) %>%
  mutate(BsmtFin.Type.2 = relevel(BsmtFin.Type.2, ref="None")) %>%
  mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF)  %>%   
  filter(!is.na(Bsmt.Full.Bath)) %>%
  filter(!is.na(Bsmt.Half.Bath)) %>%
  mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
  mutate(Fireplace.YN = 1*(Fireplaces>0)) %>%
  mutate(Fireplace.Qu =  factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
  mutate(Fireplace.Qu = relevel(Fireplace.Qu, ref="None")) %>%
  mutate(Garage.YN = 1*(!is.na(Garage.Cond))) %>%
  mutate(Garage.Type =  factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
  mutate(Garage.Type = relevel(Garage.Type, ref="None")) %>%
  mutate(Garage.Finish =  factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
  mutate(Garage.Finish = relevel(Garage.Finish, ref="None")) %>%
  mutate(Garage.Qual =  factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
  mutate(Garage.Qual = relevel(Garage.Qual, ref="None")) %>%
  mutate(Garage.Cond =  factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
  mutate(Garage.Cond = relevel(Garage.Cond, ref="None")) %>%
  mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
  mutate(Pool.YN = 1*(Pool.Area>0)) %>%
  mutate(Pool.QC =  factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
  mutate(Pool.QC = relevel(Pool.QC, ref="None")) %>%
  mutate(Fence =  factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
  mutate(Misc.Feature =  factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
  mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
  mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
  dplyr::select(-Garage.Yr.Blt) %>%
  mutate(Condition.1 = as.character(Condition.1)) %>%
  mutate(Kitchen.Qual=plyr::mapvalues(Kitchen.Qual, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
  mutate(Kitchen.Qual = as.numeric(as.character(Kitchen.Qual))) %>%
  mutate(Heating.QC=plyr::mapvalues(Heating.QC, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
  mutate(Heating.QC = as.numeric(as.character(Heating.QC))) %>%
  mutate(Bsmt.Qual = droplevels(Bsmt.Qual)) %>%
  mutate(Functional = droplevels(Functional)) %>%
  mutate(Roof.Matl = droplevels(Roof.Matl))
# Simplify Condition 1 (Park, Rail, Normal)
ind_rail<-which(data$Condition.1=="RRNn" | data$Condition.1=="RRAn" | data$Condition.1=="RRNe" | data$Condition.1=="RRAe")
ind_park<-which(data$Condition.1=="PosN" | data$Condition.1=="PosA")
data$Condition.1[ind_rail]<-"Rail"
data$Condition.1[ind_park]<-"Park"
data = data %>% 
  mutate(Condition.1 = factor(Condition.1)) %>%
  mutate(Condition.1 = relevel(Condition.1, ref="Norm")) 
# Eliminate the one entry in 'Exposure' that had been left completely empty
data_train<-data
data_train$Bsmt.Exposure[which(data_train$Bsmt.Exposure=="")]<-"None"
data_train$Bsmt.Exposure<-droplevels(data_train$Bsmt.Exposure)
data_train$Pool.Area<-data_train$Pool.Area+1
data_train$Total.Bsmt.SF<-data_train$Total.Bsmt.SF+1

The Neighborhood variable, typically of little interest other than to model the location effect, may be of more relevance when used with the map.

We are restricting attention to just the “normal sales” condition.

Part I: Simple Model

In the first model you are allowed only limited manipulations of the original data set to predict the sales price price. You are allowed to take power transformations of the original variables [square roots, logs, inverses, squares, etc.] but you are NOT allowed to create interaction variables. This means that a variable may only be used once in an equation [if you use $ x^2$ don’t use \(x\)]. Additionally, you may eliminate any data points you deem unfit. This model should have a minimum r-square of 73% (in the original units) and contain at least 6 variables but fewer than 20.

### perfromance evlaution function
performance<- function(Y, Yhat){
  
  bias<- mean(Y-Yhat[,1])
  
  max.dev<-max(abs(Y-Yhat[,1]))
  
  mean.dev<-mean(abs(Y-Yhat[,1]))
  
  RMSE<-sqrt(mean((Y-Yhat[,1])^2))
  
  
  coverage<-mean((Y>Yhat[,2]) & (Y<Yhat[,3]))
  
  out<-data.frame(bias=bias, max.dev=max.dev, mean.dev=mean.dev, RMES=RMSE, Coverage=coverage)
  
  return(out)
}
library(MASS)

Attaching package: ‘MASS’

The following object is masked from ‘package:dplyr’:

    select
# Base model with transformed predictors 
 model=lm(price ~ MS.SubClass + MS.Zoning + log(Lot.Frontage) + log(Lot.Area) +  Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1  + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge +   Roof.Style + Roof.Matl + Exterior.1st + Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Cond + Exter.Qual +   Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + Total.Bsmt.SF + Heating + Heating.QC + Central.Air + Electrical + log(X12.SF) + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + Garage.Finish + Garage.Cars + Garage.Cond + Garage.Qual + Paved.Drive + log(1+Pool.Area) + Pool.QC + Fence + Misc.Val + Mo.Sold  +Yr.Sold + Sale.Type  + TotalSq, data=data_train)
# Boxcox (indicates that log is decent)
l<-boxcox(model)

expo<-round(l$x[which.max(l$y)],2)
## Current model
## log(Lot.Frontage) currently removed to have more data points (was not included when left in the model with BIC)
model.0=lm(log(price) ~ MS.SubClass + MS.Zoning  + log(Lot.Area) +  Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1  + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge +   Roof.Style + Roof.Matl + Exterior.1st +  Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Cond + Exter.Qual +   Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + log(Total.Bsmt.SF) + Bsmt.YN+ Heating + Heating.QC + Central.Air  + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + Garage.Finish + Garage.Cars +Garage.Qual+Garage.Cond  + Paved.Drive + log(Pool.Area) + Pool.QC + Fence + Misc.Val + Mo.Sold  +Yr.Sold + Sale.Type  + log(TotalSq) + Pool.YN , data=data_train)
# There are some perfect collinearities in this model -> eliminate it via AIC/BIC
summary(model.0)

Call:
lm(formula = log(price) ~ MS.SubClass + MS.Zoning + log(Lot.Area) + 
    Street + Alley + Lot.Shape + Land.Contour + Lot.Config + 
    Land.Slope + Neighborhood + Condition.1 + Bldg.Type + House.Style + 
    Overall.Qual + Overall.Cond + HouseAge + Roof.Style + Roof.Matl + 
    Exterior.1st + Mas.Vnr.Type + log(1 + Mas.Vnr.Area) + Exter.Cond + 
    Exter.Qual + Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + 
    log(Total.Bsmt.SF) + Bsmt.YN + Heating + Heating.QC + Central.Air + 
    log(1 + Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + 
    Garage.Finish + Garage.Cars + Garage.Qual + Garage.Cond + 
    Paved.Drive + log(Pool.Area) + Pool.QC + Fence + Misc.Val + 
    Mo.Sold + Yr.Sold + Sale.Type + log(TotalSq) + Pool.YN, data = data_train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.43874 -0.04567  0.00024  0.04925  0.25557 

Coefficients: (8 not defined because of singularities)
                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)               7.135e+00  2.287e-01  31.199  < 2e-16 ***
MS.SubClass30            -7.720e-02  1.621e-02  -4.762 2.13e-06 ***
MS.SubClass40            -2.625e-02  5.441e-02  -0.483 0.629528    
MS.SubClass45            -1.585e-01  1.216e-01  -1.304 0.192564    
MS.SubClass50            -2.953e-02  3.292e-02  -0.897 0.369868    
MS.SubClass60            -3.433e-02  2.747e-02  -1.250 0.211688    
MS.SubClass70            -7.417e-02  2.987e-02  -2.483 0.013157 *  
MS.SubClass75            -1.948e-02  8.094e-02  -0.241 0.809826    
MS.SubClass80            -6.554e-02  4.932e-02  -1.329 0.184105    
MS.SubClass85            -3.716e-02  3.572e-02  -1.040 0.298403    
MS.SubClass90            -2.750e-02  2.873e-02  -0.957 0.338788    
MS.SubClass120            3.104e-02  4.763e-02   0.652 0.514768    
MS.SubClass150           -1.220e-01  1.210e-01  -1.008 0.313770    
MS.SubClass160           -5.059e-02  5.922e-02  -0.854 0.393087    
MS.SubClass180           -2.540e-02  8.118e-02  -0.313 0.754415    
MS.SubClass190           -7.996e-02  9.993e-02  -0.800 0.423747    
MS.ZoningC (all)         -8.623e-02  1.100e-01  -0.784 0.433200    
MS.ZoningFV               1.407e-01  1.042e-01   1.351 0.176927    
MS.ZoningI (all)         -2.767e-02  1.291e-01  -0.214 0.830289    
MS.ZoningRH               1.659e-01  1.037e-01   1.599 0.110075    
MS.ZoningRL               1.231e-01  9.957e-02   1.236 0.216735    
MS.ZoningRM               7.809e-02  1.008e-01   0.775 0.438763    
log(Lot.Area)             9.002e-02  1.010e-02   8.912  < 2e-16 ***
StreetPave               -1.815e-02  4.429e-02  -0.410 0.682128    
AlleyPave                -1.134e-02  2.326e-02  -0.488 0.625946    
AlleyNone                 1.801e-02  1.395e-02   1.292 0.196747    
Lot.ShapeIR2             -6.732e-04  1.582e-02  -0.043 0.966069    
Lot.ShapeIR3              6.258e-03  3.092e-02   0.202 0.839656    
Lot.ShapeReg             -1.546e-03  5.964e-03  -0.259 0.795449    
Land.ContourHLS           2.953e-02  1.852e-02   1.594 0.111086    
Land.ContourLow          -5.651e-03  2.377e-02  -0.238 0.812139    
Land.ContourLvl           2.114e-02  1.375e-02   1.537 0.124571    
Lot.ConfigCulDSac         1.419e-02  1.163e-02   1.220 0.222861    
Lot.ConfigFR2            -3.046e-02  1.445e-02  -2.108 0.035210 *  
Lot.ConfigFR3            -2.514e-02  4.191e-02  -0.600 0.548733    
Lot.ConfigInside          1.273e-02  6.572e-03   1.937 0.053009 .  
Land.SlopeMod             1.605e-02  1.453e-02   1.104 0.269592    
Land.SlopeSev            -7.217e-02  5.346e-02  -1.350 0.177255    
NeighborhoodBlueste       1.012e-01  4.633e-02   2.184 0.029154 *  
NeighborhoodBrDale        1.193e-02  4.322e-02   0.276 0.782511    
NeighborhoodBrkSide       2.929e-02  3.587e-02   0.817 0.414264    
NeighborhoodClearCr       1.518e-02  3.742e-02   0.406 0.685092    
NeighborhoodCollgCr      -4.175e-03  3.026e-02  -0.138 0.890275    
NeighborhoodCrawfor       6.731e-02  3.434e-02   1.960 0.050179 .  
NeighborhoodEdwards      -8.750e-02  3.226e-02  -2.712 0.006778 ** 
NeighborhoodGilbert      -4.620e-02  3.155e-02  -1.464 0.143425    
NeighborhoodGreens        5.330e-02  4.491e-02   1.187 0.235555    
NeighborhoodGrnHill       4.587e-01  7.033e-02   6.522 9.94e-11 ***
NeighborhoodIDOTRR       -4.480e-02  3.976e-02  -1.127 0.260077    
NeighborhoodLandmrk      -8.096e-02  9.648e-02  -0.839 0.401562    
NeighborhoodMeadowV      -9.830e-02  4.989e-02  -1.970 0.049004 *  
NeighborhoodMitchel      -1.939e-02  3.247e-02  -0.597 0.550505    
NeighborhoodNAmes        -5.754e-02  3.165e-02  -1.818 0.069271 .  
NeighborhoodNoRidge       7.880e-02  3.380e-02   2.332 0.019881 *  
NeighborhoodNPkVill       4.515e-02  4.172e-02   1.082 0.279425    
NeighborhoodNridgHt       3.993e-02  3.232e-02   1.235 0.216929    
NeighborhoodNWAmes       -3.705e-02  3.264e-02  -1.135 0.256547    
NeighborhoodOldTown      -5.117e-02  3.647e-02  -1.403 0.160835    
NeighborhoodSawyer       -1.697e-02  3.244e-02  -0.523 0.600886    
NeighborhoodSawyerW      -3.472e-02  3.147e-02  -1.103 0.270103    
NeighborhoodSomerst       7.220e-02  4.009e-02   1.801 0.071939 .  
NeighborhoodStoneBr       7.764e-02  3.451e-02   2.250 0.024620 *  
NeighborhoodSWISU        -4.571e-02  3.748e-02  -1.220 0.222869    
NeighborhoodTimber       -2.846e-02  3.383e-02  -0.841 0.400336    
NeighborhoodVeenker       2.246e-02  3.973e-02   0.565 0.571929    
Condition.1Artery        -7.044e-02  1.535e-02  -4.589 4.90e-06 ***
Condition.1Feedr         -7.402e-02  1.141e-02  -6.486 1.25e-10 ***
Condition.1Park          -2.670e-04  1.739e-02  -0.015 0.987752    
Condition.1Rail          -5.997e-02  1.480e-02  -4.052 5.38e-05 ***
Bldg.Type2fmCon           3.790e-02  9.699e-02   0.391 0.696031    
Bldg.TypeDuplex                  NA         NA      NA       NA    
Bldg.TypeTwnhs           -4.101e-02  4.983e-02  -0.823 0.410662    
Bldg.TypeTwnhsE          -3.401e-02  4.657e-02  -0.730 0.465340    
House.Style1.5Unf         1.452e-01  1.200e-01   1.210 0.226482    
House.Style1Story         4.451e-02  3.137e-02   1.419 0.156180    
House.Style2.5Fin        -8.497e-02  9.804e-02  -0.867 0.386254    
House.Style2.5Unf         4.834e-03  8.411e-02   0.057 0.954181    
House.Style2Story         4.421e-02  2.947e-02   1.500 0.133822    
House.StyleSFoyer         9.402e-02  4.126e-02   2.279 0.022844 *  
House.StyleSLvl           1.099e-01  5.280e-02   2.082 0.037563 *  
Overall.Qual              4.386e-02  3.647e-03  12.025  < 2e-16 ***
Overall.Cond              3.438e-02  3.001e-03  11.455  < 2e-16 ***
HouseAge                 -5.985e-04  1.928e-04  -3.104 0.001954 ** 
Roof.StyleGable          -6.870e-02  5.100e-02  -1.347 0.178219    
Roof.StyleGambrel        -1.341e-01  5.823e-02  -2.303 0.021454 *  
Roof.StyleHip            -7.569e-02  5.137e-02  -1.473 0.140909    
Roof.StyleMansard        -9.772e-02  6.256e-02  -1.562 0.118568    
Roof.StyleShed           -1.860e-02  9.228e-02  -0.202 0.840325    
Roof.MatlMembran          8.715e-02  1.139e-01   0.765 0.444332    
Roof.MatlRoll             7.002e-02  9.408e-02   0.744 0.456888    
Roof.MatlTar&Grv          3.100e-02  3.754e-02   0.826 0.408998    
Roof.MatlWdShake          1.155e-02  4.850e-02   0.238 0.811768    
Roof.MatlWdShngl          7.904e-02  5.041e-02   1.568 0.117148    
Exterior.1stAsphShn      -3.774e-02  9.304e-02  -0.406 0.685059    
Exterior.1stBrkComm       1.197e-01  7.003e-02   1.710 0.087573 .  
Exterior.1stBrkFace       6.513e-02  2.716e-02   2.398 0.016616 *  
Exterior.1stCBlock               NA         NA      NA       NA    
Exterior.1stCemntBd       4.768e-02  2.816e-02   1.693 0.090731 .  
Exterior.1stHdBoard       1.592e-02  2.409e-02   0.661 0.508939    
Exterior.1stImStucc      -1.087e-02  9.163e-02  -0.119 0.905605    
Exterior.1stMetalSd       2.877e-02  2.355e-02   1.222 0.222118    
Exterior.1stPlywood       9.545e-03  2.532e-02   0.377 0.706277    
Exterior.1stPreCast       3.289e-01  1.013e-01   3.245 0.001204 ** 
Exterior.1stStucco        1.632e-02  3.006e-02   0.543 0.587327    
Exterior.1stVinylSd       3.341e-02  2.395e-02   1.395 0.163211    
Exterior.1stWd Sdng       1.428e-02  2.354e-02   0.607 0.544233    
Exterior.1stWdShing       2.649e-02  2.844e-02   0.932 0.351759    
Mas.Vnr.TypeBrkFace       5.390e-02  2.675e-02   2.015 0.044108 *  
Mas.Vnr.TypeNone          1.103e-01  3.638e-02   3.031 0.002488 ** 
Mas.Vnr.TypeStone         5.822e-02  2.849e-02   2.044 0.041189 *  
log(1 + Mas.Vnr.Area)     1.221e-02  5.113e-03   2.387 0.017131 *  
Exter.CondFa             -2.636e-02  4.346e-02  -0.606 0.544314    
Exter.CondGd              2.375e-02  3.777e-02   0.629 0.529609    
Exter.CondPo             -1.132e-01  1.012e-01  -1.119 0.263270    
Exter.CondTA              3.107e-02  3.777e-02   0.823 0.410846    
Exter.QualFa             -5.306e-02  3.688e-02  -1.439 0.150478    
Exter.QualGd             -5.349e-02  1.970e-02  -2.715 0.006716 ** 
Exter.QualTA             -5.260e-02  2.179e-02  -2.414 0.015904 *  
FoundationCBlock          2.047e-02  1.118e-02   1.831 0.067407 .  
FoundationPConc           4.603e-02  1.188e-02   3.873 0.000113 ***
FoundationSlab           -6.749e-03  3.072e-02  -0.220 0.826142    
FoundationStone           4.719e-02  4.164e-02   1.133 0.257252    
FoundationWood            5.419e-02  5.410e-02   1.002 0.316657    
Bsmt.QualEx              -5.934e-01  1.230e-01  -4.826 1.56e-06 ***
Bsmt.QualFa              -6.228e-01  1.225e-01  -5.084 4.25e-07 ***
Bsmt.QualGd              -6.587e-01  1.224e-01  -5.383 8.69e-08 ***
Bsmt.QualPo              -1.649e-01  1.733e-01  -0.952 0.341390    
Bsmt.QualTA              -6.586e-01  1.221e-01  -5.395 8.14e-08 ***
Bsmt.CondEx              -5.374e-04  6.234e-02  -0.009 0.993124    
Bsmt.CondFa              -2.399e-02  1.472e-02  -1.630 0.103428    
Bsmt.CondGd               1.951e-03  1.303e-02   0.150 0.881017    
Bsmt.CondPo               8.582e-02  7.715e-02   1.112 0.266197    
Bsmt.CondTA                      NA         NA      NA       NA    
Bsmt.ExposureAv           1.326e-01  8.588e-02   1.544 0.122909    
Bsmt.ExposureGd           1.633e-01  8.631e-02   1.892 0.058769 .  
Bsmt.ExposureMn           9.901e-02  8.612e-02   1.150 0.250533    
Bsmt.ExposureNo           1.089e-01  8.585e-02   1.269 0.204848    
log(Total.Bsmt.SF)        9.271e-02  1.217e-02   7.616 5.06e-14 ***
Bsmt.YN                          NA         NA      NA       NA    
HeatingGasA               7.688e-02  9.347e-02   0.823 0.410924    
HeatingGasW               1.523e-01  9.678e-02   1.574 0.115838    
HeatingGrav               1.256e-02  1.146e-01   0.110 0.912736    
HeatingOthW               3.011e-02  1.146e-01   0.263 0.792809    
HeatingWall               1.054e-01  1.038e-01   1.015 0.310188    
Heating.QC                8.207e-03  3.392e-03   2.419 0.015689 *  
Central.AirY              5.164e-02  1.337e-02   3.862 0.000118 ***
log(1 + Low.Qual.Fin.SF)  1.195e-02  4.013e-03   2.977 0.002968 ** 
Baths                     4.982e-02  4.577e-03  10.884  < 2e-16 ***
Bedroom.AbvGr            -1.038e-02  4.588e-03  -2.263 0.023781 *  
Kitchen.AbvGr            -1.030e-01  2.454e-02  -4.199 2.87e-05 ***
Kitchen.Qual              3.426e-02  5.840e-03   5.866 5.66e-09 ***
FunctionalMaj2           -2.494e-01  5.193e-02  -4.804 1.74e-06 ***
FunctionalMin1           -1.625e-02  3.566e-02  -0.456 0.648641    
FunctionalMin2           -1.274e-02  3.606e-02  -0.353 0.723946    
FunctionalMod            -3.512e-02  3.942e-02  -0.891 0.373152    
FunctionalTyp             4.746e-02  3.290e-02   1.443 0.149399    
Fireplaces                2.516e-02  8.781e-03   2.865 0.004240 ** 
Fireplace.QuEx            1.185e-02  2.345e-02   0.505 0.613479    
Fireplace.QuFa           -3.809e-03  1.740e-02  -0.219 0.826753    
Fireplace.QuGd            8.029e-03  1.221e-02   0.658 0.510907    
Fireplace.QuPo            1.114e-02  2.043e-02   0.545 0.585585    
Fireplace.QuTA            1.034e-03  1.211e-02   0.085 0.931945    
Garage.Type2Types        -2.409e-02  3.354e-02  -0.718 0.472872    
Garage.TypeAttchd         7.027e-03  1.677e-02   0.419 0.675292    
Garage.TypeBasment       -1.302e-02  3.140e-02  -0.415 0.678518    
Garage.TypeBuiltIn        1.970e-02  1.991e-02   0.989 0.322625    
Garage.TypeCarPort       -7.919e-02  5.498e-02  -1.440 0.150039    
Garage.TypeDetchd         1.634e-02  1.637e-02   0.998 0.318665    
Garage.Finish            -7.485e-02  1.519e-01  -0.493 0.622210    
Garage.FinishFin          1.826e-02  8.378e-03   2.180 0.029443 *  
Garage.FinishRFn         -5.550e-03  7.428e-03  -0.747 0.455052    
Garage.FinishUnf                 NA         NA      NA       NA    
Garage.Cars               3.427e-02  5.610e-03   6.109 1.33e-09 ***
Garage.QualFa            -1.092e-02  1.439e-02  -0.759 0.448201    
Garage.QualGd             4.510e-02  2.910e-02   1.550 0.121430    
Garage.QualPo            -3.117e-01  7.767e-02  -4.013 6.35e-05 ***
Garage.QualTA                    NA         NA      NA       NA    
Garage.CondFa            -6.362e-02  1.804e-02  -3.527 0.000435 ***
Garage.CondGd             6.926e-04  4.160e-02   0.017 0.986719    
Garage.CondPo             7.258e-02  4.399e-02   1.650 0.099151 .  
Garage.CondTA                    NA         NA      NA       NA    
Paved.DriveP             -1.649e-02  1.865e-02  -0.884 0.376870    
Paved.DriveY              2.848e-02  1.229e-02   2.318 0.020627 *  
log(Pool.Area)           -2.200e-01  2.224e-01  -0.989 0.322688    
Pool.QCEx                 1.317e+00  1.152e+00   1.143 0.253103    
Pool.QCFa                 1.404e+00  1.416e+00   0.991 0.321709    
Pool.QCGd                 1.446e+00  1.447e+00   0.999 0.317933    
Pool.QCTA                 1.684e+00  1.362e+00   1.236 0.216777    
FenceGdWo                 1.227e-03  1.707e-02   0.072 0.942711    
FenceMnPrv               -1.599e-02  1.353e-02  -1.182 0.237505    
FenceMnWw                -4.282e-02  3.570e-02  -1.199 0.230655    
FenceNone                -1.278e-02  1.234e-02  -1.036 0.300596    
Misc.Val                  8.128e-07  4.966e-06   0.164 0.870024    
Mo.Sold2                 -9.601e-03  1.705e-02  -0.563 0.573370    
Mo.Sold3                 -1.316e-02  1.525e-02  -0.863 0.388463    
Mo.Sold4                  1.720e-02  1.479e-02   1.163 0.245173    
Mo.Sold5                  1.143e-02  1.415e-02   0.808 0.419406    
Mo.Sold6                  1.087e-02  1.386e-02   0.785 0.432847    
Mo.Sold7                  1.317e-02  1.400e-02   0.941 0.347126    
Mo.Sold8                  1.980e-03  1.537e-02   0.129 0.897511    
 [ reached getOption("max.print") -- omitted 18 rows ]
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.08396 on 1283 degrees of freedom
Multiple R-squared:  0.9562,    Adjusted R-squared:  0.9491 
F-statistic:   134 on 209 and 1283 DF,  p-value: < 2.2e-16
plot(model.0)
not plotting observations with leverage one:
  47, 168, 242, 284, 411, 638, 655, 739, 804, 990, 1011, 1194, 1198, 1324, 1344, 1376

not plotting observations with leverage one:
  47, 168, 242, 284, 411, 638, 655, 739, 804, 990, 1011, 1194, 1198, 1324, 1344, 1376

NaNs producedNaNs produced

#AIC
#model.AIC=step(model.0, k=2)
#summary(model.AIC)
#plot(model.AIC)
#BIC
#model.BIC=step(model.0, k=log(nrow(data_train)))
model.BIC=lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
               Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
               Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
               Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
               Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
               log(TotalSq), data = data_train)
summary(model.BIC)

Call:
lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
    Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
    Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
    Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
    log(TotalSq), data = data_train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.66831 -0.05150  0.00025  0.05629  0.32251 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)    
(Intercept)          7.3926554  0.1169000  63.239  < 2e-16 ***
log(Lot.Area)        0.1026436  0.0075547  13.587  < 2e-16 ***
NeighborhoodBlueste  0.0146631  0.0419500   0.350 0.726738    
NeighborhoodBrDale  -0.0634618  0.0347412  -1.827 0.067954 .  
NeighborhoodBrkSide -0.0219612  0.0306986  -0.715 0.474491    
NeighborhoodClearCr  0.0028061  0.0338084   0.083 0.933862    
NeighborhoodCollgCr -0.0131967  0.0275606  -0.479 0.632136    
NeighborhoodCrawfor  0.0479177  0.0311068   1.540 0.123679    
NeighborhoodEdwards -0.1000400  0.0294309  -3.399 0.000695 ***
NeighborhoodGilbert -0.0379098  0.0287188  -1.320 0.187036    
NeighborhoodGreens   0.0359676  0.0442891   0.812 0.416864    
NeighborhoodGrnHill  0.4397167  0.0709210   6.200 7.38e-10 ***
NeighborhoodIDOTRR  -0.1451682  0.0324860  -4.469 8.49e-06 ***
NeighborhoodLandmrk -0.0681952  0.0949941  -0.718 0.472943    
NeighborhoodMeadowV -0.1266187  0.0383000  -3.306 0.000970 ***
NeighborhoodMitchel -0.0368471  0.0295088  -1.249 0.211987    
NeighborhoodNAmes   -0.0565614  0.0285542  -1.981 0.047801 *  
NeighborhoodNoRidge  0.0623060  0.0303150   2.055 0.040034 *  
NeighborhoodNPkVill -0.0208120  0.0384833  -0.541 0.588726    
NeighborhoodNridgHt  0.0509080  0.0290237   1.754 0.079643 .  
NeighborhoodNWAmes  -0.0544560  0.0297828  -1.828 0.067693 .  
NeighborhoodOldTown -0.1188641  0.0294370  -4.038 5.68e-05 ***
NeighborhoodSawyer  -0.0229563  0.0298927  -0.768 0.442639    
NeighborhoodSawyerW -0.0515292  0.0289758  -1.778 0.075559 .  
NeighborhoodSomerst  0.0566938  0.0276847   2.048 0.040759 *  
NeighborhoodStoneBr  0.0745779  0.0321385   2.321 0.020453 *  
NeighborhoodSWISU   -0.0604508  0.0340779  -1.774 0.076293 .  
NeighborhoodTimber  -0.0324647  0.0317804  -1.022 0.307176    
NeighborhoodVeenker -0.0167331  0.0376356  -0.445 0.656671    
Condition.1Artery   -0.0760394  0.0151294  -5.026 5.64e-07 ***
Condition.1Feedr    -0.0767356  0.0113456  -6.763 1.96e-11 ***
Condition.1Park      0.0069725  0.0182004   0.383 0.701706    
Condition.1Rail     -0.0496206  0.0149453  -3.320 0.000922 ***
Overall.Qual         0.0494652  0.0034280  14.430  < 2e-16 ***
Overall.Cond         0.0351455  0.0027881  12.605  < 2e-16 ***
HouseAge            -0.0008503  0.0001883  -4.516 6.84e-06 ***
FoundationCBlock     0.0583435  0.0105020   5.555 3.30e-08 ***
FoundationPConc      0.0718433  0.0115567   6.217 6.66e-10 ***
FoundationSlab       0.0630207  0.0291017   2.166 0.030512 *  
FoundationStone      0.0038723  0.0390916   0.099 0.921107    
FoundationWood       0.0465776  0.0554929   0.839 0.401418    
Bsmt.QualEx         -0.7570306  0.1136937  -6.659 3.94e-11 ***
Bsmt.QualFa         -0.8357544  0.1124732  -7.431 1.85e-13 ***
Bsmt.QualGd         -0.8407558  0.1123115  -7.486 1.24e-13 ***
Bsmt.QualPo         -0.8411315  0.1443930  -5.825 7.04e-09 ***
Bsmt.QualTA         -0.8503101  0.1122157  -7.577 6.31e-14 ***
Bsmt.ExposureAv      0.1590409  0.0918543   1.731 0.083588 .  
Bsmt.ExposureGd      0.1929930  0.0921545   2.094 0.036415 *  
Bsmt.ExposureMn      0.1151061  0.0920620   1.250 0.211392    
Bsmt.ExposureNo      0.1296663  0.0917945   1.413 0.158000    
log(Total.Bsmt.SF)   0.1191674  0.0091824  12.978  < 2e-16 ***
Heating.QC           0.0091548  0.0033638   2.722 0.006577 ** 
Central.AirY         0.0587416  0.0121548   4.833 1.49e-06 ***
Baths                0.0470497  0.0045251  10.397  < 2e-16 ***
Bedroom.AbvGr       -0.0140992  0.0042615  -3.308 0.000961 ***
Kitchen.AbvGr       -0.0977644  0.0140392  -6.964 5.05e-12 ***
Kitchen.Qual         0.0346078  0.0056027   6.177 8.51e-10 ***
FunctionalMaj2      -0.1661268  0.0518564  -3.204 0.001387 ** 
FunctionalMin1       0.0143539  0.0345741   0.415 0.678085    
FunctionalMin2       0.0352368  0.0342227   1.030 0.303356    
FunctionalMod       -0.0059146  0.0372751  -0.159 0.873948    
FunctionalTyp        0.0882264  0.0312921   2.819 0.004877 ** 
Fireplaces           0.0278606  0.0047452   5.871 5.37e-09 ***
Garage.Cars          0.0400643  0.0046975   8.529  < 2e-16 ***
Paved.DriveP         0.0101521  0.0181899   0.558 0.576854    
Paved.DriveY         0.0573398  0.0114111   5.025 5.67e-07 ***
log(Pool.Area)       0.0175609  0.0059007   2.976 0.002969 ** 
log(TotalSq)         0.3701184  0.0154033  24.029  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.09123 on 1425 degrees of freedom
Multiple R-squared:  0.9426,    Adjusted R-squared:  0.9399 
F-statistic:   349 on 67 and 1425 DF,  p-value: < 2.2e-16
plot(model.BIC)
not plotting observations with leverage one:
  638, 655, 990

not plotting observations with leverage one:
  638, 655, 990

modelinteract = lm(log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual:Baths:Neighborhood + Overall.Qual:Baths + Overall.Qual:Neighborhood + Overall.Qual + Baths + Neighborhood + Overall.Qual:Baths:log(Total.Bsmt.SF) + Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars:log(Total.Bsmt.SF):log(TotalSq) + Overall.Qual:Garage.Cars:log(TotalSq) + Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + Garage.Cars:log(Total.Bsmt.SF):log(TotalSq) + Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area), data = data_train)
model.interact.reduced=lm(log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + 
    Baths + Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + 
    log(TotalSq) + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
    Bsmt.Exposure + Heating.QC + Central.Air + Bedroom.AbvGr + 
    Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + 
    Paved.Drive + Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + 
    Garage.Cars:log(TotalSq) + Overall.Qual:Garage.Cars:log(TotalSq), 
    data = data_train)
summary(model.interact.reduced)

Call:
lm(formula = log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + 
    Baths + Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + 
    log(TotalSq) + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
    Bsmt.Exposure + Heating.QC + Central.Air + Bedroom.AbvGr + 
    Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + 
    Paved.Drive + Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + 
    Garage.Cars:log(TotalSq) + Overall.Qual:Garage.Cars:log(TotalSq), 
    data = data_train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.66963 -0.05131  0.00028  0.05415  0.31533 

Coefficients:
                                        Estimate Std. Error t value Pr(>|t|)    
(Intercept)                            6.5102614  0.4057490  16.045  < 2e-16 ***
log(Lot.Area)                          0.0987836  0.0075415  13.099  < 2e-16 ***
Condition.1Artery                     -0.0736717  0.0150310  -4.901 1.06e-06 ***
Condition.1Feedr                      -0.0745876  0.0112824  -6.611 5.39e-11 ***
Condition.1Park                       -0.0051531  0.0181031  -0.285 0.775950    
Condition.1Rail                       -0.0456100  0.0148189  -3.078 0.002125 ** 
Overall.Qual                           0.2068733  0.0752569   2.749 0.006055 ** 
Baths                                  0.0468993  0.0045396  10.331  < 2e-16 ***
NeighborhoodBlueste                    0.0203947  0.0416095   0.490 0.624106    
NeighborhoodBrDale                    -0.0722602  0.0345132  -2.094 0.036463 *  
NeighborhoodBrkSide                   -0.0217135  0.0304547  -0.713 0.475977    
NeighborhoodClearCr                    0.0103092  0.0336539   0.306 0.759399    
NeighborhoodCollgCr                   -0.0159676  0.0273774  -0.583 0.559823    
NeighborhoodCrawfor                    0.0499783  0.0308951   1.618 0.105954    
NeighborhoodEdwards                   -0.1032780  0.0291913  -3.538 0.000416 ***
NeighborhoodGilbert                   -0.0368491  0.0285011  -1.293 0.196256    
NeighborhoodGreens                     0.0574166  0.0441283   1.301 0.193425    
NeighborhoodGrnHill                    0.4438689  0.0705238   6.294 4.11e-10 ***
NeighborhoodIDOTRR                    -0.1470244  0.0322360  -4.561 5.53e-06 ***
NeighborhoodLandmrk                   -0.0700365  0.0941311  -0.744 0.456981    
NeighborhoodMeadowV                   -0.1441208  0.0380517  -3.787 0.000159 ***
NeighborhoodMitchel                   -0.0346045  0.0293105  -1.181 0.237952    
NeighborhoodNAmes                     -0.0586303  0.0283935  -2.065 0.039111 *  
NeighborhoodNoRidge                    0.0250677  0.0307515   0.815 0.415112    
NeighborhoodNPkVill                   -0.0218909  0.0381536  -0.574 0.566222    
NeighborhoodNridgHt                    0.0317326  0.0289648   1.096 0.273459    
NeighborhoodNWAmes                    -0.0465829  0.0295620  -1.576 0.115302    
NeighborhoodOldTown                   -0.1206424  0.0292366  -4.126 3.90e-05 ***
NeighborhoodSawyer                    -0.0277534  0.0297174  -0.934 0.350509    
NeighborhoodSawyerW                   -0.0496333  0.0287674  -1.725 0.084685 .  
NeighborhoodSomerst                    0.0576101  0.0274356   2.100 0.035919 *  
NeighborhoodStoneBr                    0.0743440  0.0318815   2.332 0.019846 *  
NeighborhoodSWISU                     -0.0658180  0.0339123  -1.941 0.052476 .  
NeighborhoodTimber                    -0.0245197  0.0315509  -0.777 0.437201    
NeighborhoodVeenker                   -0.0060078  0.0372772  -0.161 0.871985    
Garage.Cars                            1.2032426  0.2341509   5.139 3.15e-07 ***
log(Total.Bsmt.SF)                     0.1189555  0.0091090  13.059  < 2e-16 ***
log(TotalSq)                           0.5012898  0.0581418   8.622  < 2e-16 ***
Overall.Cond                           0.0336221  0.0027929  12.039  < 2e-16 ***
HouseAge                              -0.0009157  0.0001870  -4.896 1.09e-06 ***
FoundationCBlock                       0.0589952  0.0104102   5.667 1.76e-08 ***
FoundationPConc                        0.0731980  0.0114720   6.381 2.38e-10 ***
FoundationSlab                         0.0718932  0.0289054   2.487 0.012990 *  
FoundationStone                        0.0044701  0.0389376   0.115 0.908619    
FoundationWood                         0.0392228  0.0550548   0.712 0.476314    
Bsmt.QualEx                           -0.7632015  0.1127644  -6.768 1.90e-11 ***
Bsmt.QualFa                           -0.8262069  0.1114845  -7.411 2.14e-13 ***
Bsmt.QualGd                           -0.8267350  0.1113854  -7.422 1.97e-13 ***
Bsmt.QualPo                           -0.8358514  0.1430638  -5.843 6.36e-09 ***
Bsmt.QualTA                           -0.8376454  0.1112764  -7.528 9.13e-14 ***
Bsmt.ExposureAv                        0.1575361  0.0910068   1.731 0.083662 .  
Bsmt.ExposureGd                        0.1836363  0.0913334   2.011 0.044555 *  
Bsmt.ExposureMn                        0.1122697  0.0912138   1.231 0.218586    
Bsmt.ExposureNo                        0.1280950  0.0909443   1.408 0.159202    
Heating.QC                             0.0093065  0.0033440   2.783 0.005457 ** 
Central.AirY                           0.0636338  0.0121613   5.232 1.92e-07 ***
Bedroom.AbvGr                         -0.0147371  0.0042642  -3.456 0.000564 ***
Kitchen.AbvGr                         -0.0910776  0.0142064  -6.411 1.96e-10 ***
Kitchen.Qual                           0.0332874  0.0055725   5.973 2.93e-09 ***
FunctionalMaj2                        -0.1605476  0.0514362  -3.121 0.001837 ** 
FunctionalMin1                         0.0184235  0.0343702   0.536 0.592021    
FunctionalMin2                         0.0407226  0.0339893   1.198 0.231077    
FunctionalMod                          0.0029487  0.0370700   0.080 0.936611    
FunctionalTyp                          0.0888654  0.0310927   2.858 0.004324 ** 
Fireplaces                             0.0296062  0.0047109   6.285 4.36e-10 ***
Paved.DriveP                           0.0171341  0.0181069   0.946 0.344168    
Paved.DriveY                           0.0605392  0.0114220   5.300 1.34e-07 ***
Overall.Qual:Garage.Cars              -0.1928486  0.0384268  -5.019 5.86e-07 ***
Overall.Qual:log(TotalSq)             -0.0225185  0.0105342  -2.138 0.032715 *  
Garage.Cars:log(TotalSq)              -0.1633273  0.0325288  -5.021 5.79e-07 ***
Overall.Qual:Garage.Cars:log(TotalSq)  0.0269148  0.0052360   5.140 3.12e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.09038 on 1422 degrees of freedom
Multiple R-squared:  0.9437,    Adjusted R-squared:  0.941 
F-statistic: 340.7 on 70 and 1422 DF,  p-value: < 2.2e-16
plot(model.interact.reduced)
not plotting observations with leverage one:
  638, 655, 990

not plotting observations with leverage one:
  638, 655, 990

step(modelinteract, k=log(nrow(data_train)))
Start:  AIC=-6393.34
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual:Baths:Neighborhood + 
    Overall.Qual:Baths + Overall.Qual:Neighborhood + Overall.Qual + 
    Baths + Neighborhood + Overall.Qual:Baths:log(Total.Bsmt.SF) + 
    Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars:log(Total.Bsmt.SF):log(TotalSq) + 
    Overall.Qual:Garage.Cars:log(TotalSq) + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(Total.Bsmt.SF):log(TotalSq) + 
    Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + 
    Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + Overall.Cond + 
    HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + Heating.QC + 
    Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + 
    Functional + Fireplaces + Paved.Drive + log(Pool.Area)

                                                           Df Sum of Sq    RSS     AIC
- Overall.Qual:Baths:Neighborhood                          25   0.27651 11.351 -6539.2
- Overall.Qual:log(Total.Bsmt.SF):Garage.Cars:log(TotalSq)  1   0.00141 11.076 -6400.5
- Overall.Qual:Baths:log(Total.Bsmt.SF)                     1   0.00289 11.077 -6400.3
<none>                                                                  11.075 -6393.3
- log(Pool.Area)                                            1   0.05840 11.133 -6392.8
- Heating.QC                                                1   0.07333 11.148 -6390.8
- Bedroom.AbvGr                                             1   0.07357 11.148 -6390.8
- Foundation                                                5   0.32271 11.397 -6387.0
- Bsmt.Exposure                                             4   0.32453 11.399 -6379.4
- Paved.Drive                                               2   0.25326 11.328 -6374.2
- Central.Air                                               1   0.20054 11.275 -6373.8
- HouseAge                                                  1   0.21102 11.286 -6372.5
- Kitchen.Qual                                              1   0.24251 11.317 -6368.3
- Bsmt.Qual                                                 5   0.54381 11.618 -6358.3
- Condition.1                                               4   0.48963 11.564 -6358.0
- Kitchen.AbvGr                                             1   0.33164 11.406 -6356.6
- Fireplaces                                                1   0.37225 11.447 -6351.3
- Functional                                                5   0.62194 11.696 -6348.3
- Overall.Cond                                              1   1.06255 12.137 -6263.9
- log(Lot.Area)                                             1   1.18813 12.263 -6248.5

Step:  AIC=-6539.23
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:Baths + Overall.Qual:Neighborhood + Overall.Qual:log(Total.Bsmt.SF) + 
    Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + 
    log(Total.Bsmt.SF):log(TotalSq) + Overall.Qual:Baths:log(Total.Bsmt.SF) + 
    Overall.Qual:Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):Garage.Cars:log(TotalSq) + 
    Overall.Qual:log(Total.Bsmt.SF):Garage.Cars:log(TotalSq)

                                                           Df Sum of Sq    RSS     AIC
- Overall.Qual:Neighborhood                                24   0.19443 11.545 -6689.3
- Overall.Qual:Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)  1   0.00006 11.351 -6546.5
- Overall.Qual:Baths:log(Total.Bsmt.SF)                     1   0.00263 11.354 -6546.2
- log(Pool.Area)                                            1   0.04903 11.400 -6540.1
<none>                                                                  11.351 -6539.2
- Heating.QC                                                1   0.07445 11.425 -6536.8
- Bedroom.AbvGr                                             1   0.09175 11.443 -6534.5
- Foundation                                                5   0.34270 11.694 -6531.4
- Bsmt.Exposure                                             4   0.34383 11.695 -6523.9
- Central.Air                                               1   0.18322 11.534 -6522.6
- HouseAge                                                  1   0.18603 11.537 -6522.3
- Paved.Drive                                               2   0.26057 11.612 -6520.0
- Kitchen.Qual                                              1   0.24544 11.596 -6514.6
- Fireplaces                                                1   0.33425 11.685 -6503.2
- Bsmt.Qual                                                 5   0.56586 11.917 -6503.1
- Kitchen.AbvGr                                             1   0.35655 11.707 -6500.4
- Condition.1                                               4   0.56029 11.911 -6496.5
- Functional                                                5   0.62699 11.978 -6495.5
- Overall.Cond                                              1   1.15804 12.509 -6401.5
- log(Lot.Area)                                             1   1.31673 12.668 -6382.7

Step:  AIC=-6689.28
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:Baths + Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + 
    Overall.Qual:Baths:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars:log(TotalSq) + 
    Garage.Cars:log(Total.Bsmt.SF):log(TotalSq) + Overall.Qual:Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)

                                                           Df Sum of Sq    RSS     AIC
- Overall.Qual:Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)  1   0.00034 11.546 -6696.5
- Overall.Qual:Baths:log(Total.Bsmt.SF)                     1   0.00121 11.547 -6696.4
- log(Pool.Area)                                            1   0.05614 11.602 -6689.3
<none>                                                                  11.545 -6689.3
- Heating.QC                                                1   0.06801 11.613 -6687.8
- Bedroom.AbvGr                                             1   0.09581 11.641 -6684.2
- Foundation                                                5   0.37587 11.921 -6678.0
- HouseAge                                                  1   0.19414 11.740 -6671.7
- Central.Air                                               1   0.19723 11.743 -6671.3
- Bsmt.Exposure                                             4   0.37270 11.918 -6671.1
- Paved.Drive                                               2   0.26302 11.808 -6670.3
- Kitchen.Qual                                              1   0.27526 11.821 -6661.4
- Fireplaces                                                1   0.30209 11.848 -6658.0
- Kitchen.AbvGr                                             1   0.33387 11.879 -6654.0
- Bsmt.Qual                                                 5   0.63863 12.184 -6645.4
- Functional                                                5   0.65217 12.198 -6643.8
- Condition.1                                               4   0.59422 12.140 -6643.6
- Neighborhood                                             27   2.65674 14.202 -6577.4
- Overall.Cond                                              1   1.17879 12.724 -6551.4
- log(Lot.Area)                                             1   1.36912 12.915 -6529.3

Step:  AIC=-6696.54
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:Baths + Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + 
    Overall.Qual:Baths:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars:log(TotalSq) + 
    Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)

                                              Df Sum of Sq    RSS     AIC
- Overall.Qual:Baths:log(Total.Bsmt.SF)        1   0.00191 11.548 -6703.6
- Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)  1   0.00473 11.550 -6703.2
- log(Pool.Area)                               1   0.05614 11.602 -6696.6
<none>                                                     11.546 -6696.5
- Heating.QC                                   1   0.06810 11.614 -6695.1
- Bedroom.AbvGr                                1   0.09669 11.643 -6691.4
- Foundation                                   5   0.37556 11.921 -6685.3
- Overall.Qual:Garage.Cars:log(TotalSq)        1   0.15669 11.702 -6683.7
- HouseAge                                     1   0.19387 11.740 -6679.0
- Central.Air                                  1   0.19691 11.743 -6678.6
- Bsmt.Exposure                                4   0.37501 11.921 -6678.1
- Paved.Drive                                  2   0.26427 11.810 -6677.4
- Kitchen.Qual                                 1   0.27605 11.822 -6668.6
- Fireplaces                                   1   0.30183 11.848 -6665.3
- Kitchen.AbvGr                                1   0.33539 11.881 -6661.1
- Bsmt.Qual                                    5   0.64444 12.190 -6652.0
- Functional                                   5   0.65188 12.198 -6651.1
- Condition.1                                  4   0.59396 12.140 -6650.9
- Neighborhood                                27   2.65682 14.203 -6584.7
- Overall.Cond                                 1   1.18054 12.726 -6558.5
- log(Lot.Area)                                1   1.36879 12.915 -6536.6

Step:  AIC=-6703.6
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:Baths + Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + 
    Overall.Qual:Garage.Cars:log(TotalSq) + Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)

                                              Df Sum of Sq    RSS     AIC
- Overall.Qual:Baths                           1   0.00009 11.548 -6710.9
- Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)  1   0.00435 11.552 -6710.4
- Overall.Qual:log(Total.Bsmt.SF)              1   0.00448 11.552 -6710.3
- log(Pool.Area)                               1   0.05662 11.604 -6703.6
<none>                                                     11.548 -6703.6
- Heating.QC                                   1   0.06780 11.616 -6702.2
- Bedroom.AbvGr                                1   0.09749 11.645 -6698.4
- Foundation                                   5   0.37443 11.922 -6692.5
- Overall.Qual:Garage.Cars:log(TotalSq)        1   0.16502 11.713 -6689.7
- HouseAge                                     1   0.19304 11.741 -6686.2
- Central.Air                                  1   0.19873 11.746 -6685.4
- Paved.Drive                                  2   0.26390 11.812 -6684.5
- Bsmt.Exposure                                4   0.38115 11.929 -6684.4
- Kitchen.Qual                                 1   0.27478 11.822 -6675.8
- Fireplaces                                   1   0.30256 11.850 -6672.3
- Kitchen.AbvGr                                1   0.34713 11.895 -6666.7
- Functional                                   5   0.65003 12.198 -6658.4
- Condition.1                                  4   0.59210 12.140 -6658.2
- Bsmt.Qual                                    5   0.67344 12.221 -6655.5
- Neighborhood                                27   2.65828 14.206 -6591.6
- Overall.Cond                                 1   1.20245 12.750 -6563.0
- log(Lot.Area)                                1   1.38904 12.937 -6541.3

Step:  AIC=-6710.9
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + 
    Overall.Qual:Garage.Cars:log(TotalSq) + Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)

                                              Df Sum of Sq    RSS     AIC
- Garage.Cars:log(Total.Bsmt.SF):log(TotalSq)  1   0.00429 11.552 -6717.7
- Overall.Qual:log(Total.Bsmt.SF)              1   0.00457 11.552 -6717.6
- log(Pool.Area)                               1   0.05665 11.604 -6710.9
<none>                                                     11.548 -6710.9
- Heating.QC                                   1   0.06844 11.616 -6709.4
- Bedroom.AbvGr                                1   0.09743 11.645 -6705.7
- Foundation                                   5   0.37460 11.922 -6699.8
- Overall.Qual:Garage.Cars:log(TotalSq)        1   0.17457 11.722 -6695.8
- HouseAge                                     1   0.19344 11.741 -6693.4
- Central.Air                                  1   0.20024 11.748 -6692.5
- Paved.Drive                                  2   0.26383 11.812 -6691.8
- Bsmt.Exposure                                4   0.38132 11.929 -6691.6
- Kitchen.Qual                                 1   0.27537 11.823 -6683.0
- Fireplaces                                   1   0.30385 11.852 -6679.4
- Kitchen.AbvGr                                1   0.34733 11.895 -6674.0
- Functional                                   5   0.65081 12.199 -6665.6
- Condition.1                                  4   0.59303 12.141 -6665.4
- Bsmt.Qual                                    5   0.67508 12.223 -6662.6
- Baths                                        1   0.84956 12.397 -6612.2
- Neighborhood                                27   2.65943 14.207 -6598.8
- Overall.Cond                                 1   1.20663 12.754 -6569.8
- log(Lot.Area)                                1   1.38984 12.938 -6548.5

Step:  AIC=-6717.66
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:log(Total.Bsmt.SF) + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + log(Total.Bsmt.SF):log(TotalSq) + 
    Overall.Qual:Garage.Cars:log(TotalSq)

                                        Df Sum of Sq    RSS     AIC
- Overall.Qual:log(Total.Bsmt.SF)        1   0.00482 11.557 -6724.3
- log(Total.Bsmt.SF):log(TotalSq)        1   0.00739 11.559 -6724.0
- log(Pool.Area)                         1   0.05656 11.609 -6717.7
<none>                                               11.552 -6717.7
- Heating.QC                             1   0.06773 11.620 -6716.2
- Bedroom.AbvGr                          1   0.09473 11.647 -6712.8
- Foundation                             5   0.37508 11.927 -6706.5
- Overall.Qual:Garage.Cars:log(TotalSq)  1   0.17252 11.725 -6702.8
- HouseAge                               1   0.19341 11.745 -6700.2
- Central.Air                            1   0.20225 11.754 -6699.1
- Paved.Drive                            2   0.26022 11.812 -6699.0
- Bsmt.Exposure                          4   0.38110 11.933 -6698.4
- Kitchen.Qual                           1   0.27265 11.825 -6690.1
- Fireplaces                             1   0.30489 11.857 -6686.1
- Kitchen.AbvGr                          1   0.34493 11.897 -6681.0
- Functional                             5   0.64652 12.199 -6672.9
- Condition.1                            4   0.59558 12.148 -6671.8
- Bsmt.Qual                              5   0.68287 12.235 -6668.5
- Baths                                  1   0.85824 12.410 -6618.0
- Neighborhood                          27   2.65589 14.208 -6606.0
- Overall.Cond                           1   1.21843 12.771 -6575.3
- log(Lot.Area)                          1   1.38629 12.938 -6555.8

Step:  AIC=-6724.34
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + 
    log(Total.Bsmt.SF):log(TotalSq) + Overall.Qual:Garage.Cars:log(TotalSq)

                                        Df Sum of Sq    RSS     AIC
- log(Total.Bsmt.SF):log(TotalSq)        1   0.00423 11.561 -6731.1
- log(Pool.Area)                         1   0.05580 11.613 -6724.5
<none>                                               11.557 -6724.3
- Heating.QC                             1   0.06690 11.624 -6723.0
- Bedroom.AbvGr                          1   0.09771 11.655 -6719.1
- Foundation                             5   0.37544 11.932 -6713.2
- Overall.Qual:Garage.Cars:log(TotalSq)  1   0.16899 11.726 -6710.0
- HouseAge                               1   0.19173 11.749 -6707.1
- Paved.Drive                            2   0.25664 11.813 -6706.2
- Central.Air                            1   0.20052 11.757 -6706.0
- Bsmt.Exposure                          4   0.38280 11.940 -6704.9
- Kitchen.Qual                           1   0.27739 11.834 -6696.2
- Fireplaces                             1   0.30403 11.861 -6692.9
- Kitchen.AbvGr                          1   0.34163 11.899 -6688.2
- Functional                             5   0.64326 12.200 -6680.0
- Condition.1                            4   0.59232 12.149 -6679.0
- Bsmt.Qual                              5   0.73185 12.289 -6669.2
- Baths                                  1   0.85917 12.416 -6624.6
- Neighborhood                          27   2.65109 14.208 -6613.3
- Overall.Cond                           1   1.21771 12.775 -6582.1
- log(Lot.Area)                          1   1.39657 12.953 -6561.3

Step:  AIC=-6731.1
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + log(Pool.Area) + 
    Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + 
    Overall.Qual:Garage.Cars:log(TotalSq)

                                        Df Sum of Sq    RSS     AIC
- log(Pool.Area)                         1   0.05569 11.617 -6731.2
<none>                                               11.561 -6731.1
- Heating.QC                             1   0.06618 11.627 -6729.9
- Bedroom.AbvGr                          1   0.09745 11.659 -6725.9
- Foundation                             5   0.37157 11.933 -6720.4
- HouseAge                               1   0.19214 11.753 -6713.8
- Paved.Drive                            2   0.26198 11.823 -6712.3
- Bsmt.Exposure                          4   0.38138 11.943 -6711.9
- Overall.Qual:Garage.Cars:log(TotalSq)  1   0.20815 11.769 -6711.8
- Central.Air                            1   0.21974 11.781 -6710.3
- Kitchen.Qual                           1   0.27785 11.839 -6703.0
- Fireplaces                             1   0.30489 11.866 -6699.5
- Kitchen.AbvGr                          1   0.33922 11.900 -6695.2
- Functional                             5   0.63914 12.200 -6687.3
- Condition.1                            4   0.59007 12.151 -6686.0
- Bsmt.Qual                              5   0.72871 12.290 -6676.4
- Baths                                  1   0.85871 12.420 -6631.4
- Neighborhood                          27   2.64847 14.210 -6620.5
- Overall.Cond                           1   1.21550 12.777 -6589.2
- log(Lot.Area)                          1   1.40140 12.963 -6567.6
- log(Total.Bsmt.SF)                     1   1.41469 12.976 -6566.1

Step:  AIC=-6731.24
log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + Baths + 
    Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + log(TotalSq) + 
    Overall.Cond + HouseAge + Foundation + Bsmt.Qual + Bsmt.Exposure + 
    Heating.QC + Central.Air + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Paved.Drive + Overall.Qual:Garage.Cars + 
    Overall.Qual:log(TotalSq) + Garage.Cars:log(TotalSq) + Overall.Qual:Garage.Cars:log(TotalSq)

                                        Df Sum of Sq    RSS     AIC
<none>                                               11.617 -6731.2
- Heating.QC                             1   0.06327 11.680 -6730.4
- Bedroom.AbvGr                          1   0.09757 11.714 -6726.1
- Foundation                             5   0.37226 11.989 -6720.7
- HouseAge                               1   0.19580 11.813 -6713.6
- Paved.Drive                            2   0.25832 11.875 -6713.0
- Bsmt.Exposure                          4   0.38607 12.003 -6711.7
- Overall.Qual:Garage.Cars:log(TotalSq)  1   0.21586 11.833 -6711.1
- Central.Air                            1   0.22367 11.841 -6710.1
- Kitchen.Qual                           1   0.29150 11.908 -6701.5
- Fireplaces                             1   0.32266 11.939 -6697.6
- Kitchen.AbvGr                          1   0.33577 11.953 -6696.0
- Functional                             5   0.62605 12.243 -6689.4
- Condition.1                            4   0.57680 12.194 -6688.1
- Bsmt.Qual                              5   0.71459 12.331 -6678.7
- Baths                                  1   0.87194 12.489 -6630.5
- Neighborhood                          27   2.63950 14.256 -6622.9
- Overall.Cond                           1   1.18396 12.801 -6593.6
- log(Total.Bsmt.SF)                     1   1.39321 13.010 -6569.4
- log(Lot.Area)                          1   1.40167 13.018 -6568.5

Call:
lm(formula = log(price) ~ log(Lot.Area) + Condition.1 + Overall.Qual + 
    Baths + Neighborhood + Garage.Cars + log(Total.Bsmt.SF) + 
    log(TotalSq) + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
    Bsmt.Exposure + Heating.QC + Central.Air + Bedroom.AbvGr + 
    Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + 
    Paved.Drive + Overall.Qual:Garage.Cars + Overall.Qual:log(TotalSq) + 
    Garage.Cars:log(TotalSq) + Overall.Qual:Garage.Cars:log(TotalSq), 
    data = data_train)

Coefficients:
                          (Intercept)                          log(Lot.Area)  
                            6.5102614                              0.0987836  
                    Condition.1Artery                       Condition.1Feedr  
                           -0.0736717                             -0.0745876  
                      Condition.1Park                        Condition.1Rail  
                           -0.0051531                             -0.0456100  
                         Overall.Qual                                  Baths  
                            0.2068733                              0.0468993  
                  NeighborhoodBlueste                     NeighborhoodBrDale  
                            0.0203947                             -0.0722602  
                  NeighborhoodBrkSide                    NeighborhoodClearCr  
                           -0.0217135                              0.0103092  
                  NeighborhoodCollgCr                    NeighborhoodCrawfor  
                           -0.0159676                              0.0499783  
                  NeighborhoodEdwards                    NeighborhoodGilbert  
                           -0.1032780                             -0.0368491  
                   NeighborhoodGreens                    NeighborhoodGrnHill  
                            0.0574166                              0.4438689  
                   NeighborhoodIDOTRR                    NeighborhoodLandmrk  
                           -0.1470244                             -0.0700365  
                  NeighborhoodMeadowV                    NeighborhoodMitchel  
                           -0.1441208                             -0.0346045  
                    NeighborhoodNAmes                    NeighborhoodNoRidge  
                           -0.0586303                              0.0250677  
                  NeighborhoodNPkVill                    NeighborhoodNridgHt  
                           -0.0218909                              0.0317326  
                   NeighborhoodNWAmes                    NeighborhoodOldTown  
                           -0.0465829                             -0.1206424  
                   NeighborhoodSawyer                    NeighborhoodSawyerW  
                           -0.0277534                             -0.0496333  
                  NeighborhoodSomerst                    NeighborhoodStoneBr  
                            0.0576101                              0.0743440  
                    NeighborhoodSWISU                     NeighborhoodTimber  
                           -0.0658180                             -0.0245197  
                  NeighborhoodVeenker                            Garage.Cars  
                           -0.0060078                              1.2032426  
                   log(Total.Bsmt.SF)                           log(TotalSq)  
                            0.1189555                              0.5012898  
                         Overall.Cond                               HouseAge  
                            0.0336221                             -0.0009157  
                     FoundationCBlock                        FoundationPConc  
                            0.0589952                              0.0731980  
                       FoundationSlab                        FoundationStone  
                            0.0718932                              0.0044701  
                       FoundationWood                            Bsmt.QualEx  
                            0.0392228                             -0.7632015  
                          Bsmt.QualFa                            Bsmt.QualGd  
                           -0.8262069                             -0.8267350  
                          Bsmt.QualPo                            Bsmt.QualTA  
                           -0.8358514                             -0.8376454  
                      Bsmt.ExposureAv                        Bsmt.ExposureGd  
                            0.1575361                              0.1836363  
                      Bsmt.ExposureMn                        Bsmt.ExposureNo  
                            0.1122697                              0.1280950  
                           Heating.QC                           Central.AirY  
                            0.0093065                              0.0636338  
                        Bedroom.AbvGr                          Kitchen.AbvGr  
                           -0.0147371                             -0.0910776  
                         Kitchen.Qual                         FunctionalMaj2  
                            0.0332874                             -0.1605476  
                       FunctionalMin1                         FunctionalMin2  
                            0.0184235                              0.0407226  
                        FunctionalMod                          FunctionalTyp  
                            0.0029487                              0.0888654  
                           Fireplaces                           Paved.DriveP  
                            0.0296062                              0.0171341  
                         Paved.DriveY               Overall.Qual:Garage.Cars  
                            0.0605392                             -0.1928486  
            Overall.Qual:log(TotalSq)               Garage.Cars:log(TotalSq)  
                           -0.0225185                             -0.1633273  
Overall.Qual:Garage.Cars:log(TotalSq)  
                            0.0269148  
library(tree)
tree = tree(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
               Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
               Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
               Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
               Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
               log(TotalSq), data = data_train)
# use cross-validation to prune
cv.tree = cv.tree(tree)
plot(cv.tree$size, cv.tree$dev, type='b')

bb = cv.tree$size[which.min(cv.tree$dev)]
# prune the tree for the size that is optimal based on CV
prune.tree=prune.tree(tree, best=bb)
# plot the pruned tree
plot(prune.tree)
text(prune.tree, pretty=0)

prune.tree
node), split, n, deviance, yval
      * denotes terminal node

 1) root 1493 206.500 12.01  
   2) Overall.Qual < 6.5 977  70.660 11.83  
     4) Baths < 2.25 718  42.240 11.75  
       8) Overall.Qual < 4.5 136  12.510 11.49  
        16) log(Total.Bsmt.SF) < 6.61667 74   5.731 11.34 *
        17) log(Total.Bsmt.SF) > 6.61667 62   3.197 11.67 *
       9) Overall.Qual > 4.5 582  18.830 11.81  
        18) log(Total.Bsmt.SF) < 6.89315 343   9.468 11.75 *
        19) log(Total.Bsmt.SF) > 6.89315 239   6.576 11.89 *
     5) Baths > 2.25 259  10.630 12.05  
      10) Neighborhood: Blueste,BrDale,Edwards,IDOTRR,Landmrk,MeadowV,NAmes,NPkVill,OldTown,Sawyer,SWISU 103   4.687 11.92 *
      11) Neighborhood: BrkSide,ClearCr,CollgCr,Crawfor,Gilbert,Mitchel,NoRidge,NridgHt,NWAmes,SawyerW,Somerst,Timber,Veenker 156   3.029 12.14 *
   3) Overall.Qual > 6.5 516  43.610 12.35  
     6) Garage.Cars < 2.5 382  19.670 12.25  
      12) log(TotalSq) < 7.30149 135   3.599 12.09 *
      13) log(TotalSq) > 7.30149 247  10.730 12.34  
        26) log(Total.Bsmt.SF) < 6.93537 111   2.919 12.20 *
        27) log(Total.Bsmt.SF) > 6.93537 136   4.112 12.45 *
     7) Garage.Cars > 2.5 134   9.435 12.63  
      14) log(Total.Bsmt.SF) < 7.44891 93   3.847 12.53 *
      15) log(Total.Bsmt.SF) > 7.44891 41   2.029 12.88 *
summary(model.BIC)

Call:
lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
    Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
    Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
    Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
    log(TotalSq), data = data_train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.66831 -0.05150  0.00025  0.05629  0.32251 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)    
(Intercept)          7.3926554  0.1169000  63.239  < 2e-16 ***
log(Lot.Area)        0.1026436  0.0075547  13.587  < 2e-16 ***
NeighborhoodBlueste  0.0146631  0.0419500   0.350 0.726738    
NeighborhoodBrDale  -0.0634618  0.0347412  -1.827 0.067954 .  
NeighborhoodBrkSide -0.0219612  0.0306986  -0.715 0.474491    
NeighborhoodClearCr  0.0028061  0.0338084   0.083 0.933862    
NeighborhoodCollgCr -0.0131967  0.0275606  -0.479 0.632136    
NeighborhoodCrawfor  0.0479177  0.0311068   1.540 0.123679    
NeighborhoodEdwards -0.1000400  0.0294309  -3.399 0.000695 ***
NeighborhoodGilbert -0.0379098  0.0287188  -1.320 0.187036    
NeighborhoodGreens   0.0359676  0.0442891   0.812 0.416864    
NeighborhoodGrnHill  0.4397167  0.0709210   6.200 7.38e-10 ***
NeighborhoodIDOTRR  -0.1451682  0.0324860  -4.469 8.49e-06 ***
NeighborhoodLandmrk -0.0681952  0.0949941  -0.718 0.472943    
NeighborhoodMeadowV -0.1266187  0.0383000  -3.306 0.000970 ***
NeighborhoodMitchel -0.0368471  0.0295088  -1.249 0.211987    
NeighborhoodNAmes   -0.0565614  0.0285542  -1.981 0.047801 *  
NeighborhoodNoRidge  0.0623060  0.0303150   2.055 0.040034 *  
NeighborhoodNPkVill -0.0208120  0.0384833  -0.541 0.588726    
NeighborhoodNridgHt  0.0509080  0.0290237   1.754 0.079643 .  
NeighborhoodNWAmes  -0.0544560  0.0297828  -1.828 0.067693 .  
NeighborhoodOldTown -0.1188641  0.0294370  -4.038 5.68e-05 ***
NeighborhoodSawyer  -0.0229563  0.0298927  -0.768 0.442639    
NeighborhoodSawyerW -0.0515292  0.0289758  -1.778 0.075559 .  
NeighborhoodSomerst  0.0566938  0.0276847   2.048 0.040759 *  
NeighborhoodStoneBr  0.0745779  0.0321385   2.321 0.020453 *  
NeighborhoodSWISU   -0.0604508  0.0340779  -1.774 0.076293 .  
NeighborhoodTimber  -0.0324647  0.0317804  -1.022 0.307176    
NeighborhoodVeenker -0.0167331  0.0376356  -0.445 0.656671    
Condition.1Artery   -0.0760394  0.0151294  -5.026 5.64e-07 ***
Condition.1Feedr    -0.0767356  0.0113456  -6.763 1.96e-11 ***
Condition.1Park      0.0069725  0.0182004   0.383 0.701706    
Condition.1Rail     -0.0496206  0.0149453  -3.320 0.000922 ***
Overall.Qual         0.0494652  0.0034280  14.430  < 2e-16 ***
Overall.Cond         0.0351455  0.0027881  12.605  < 2e-16 ***
HouseAge            -0.0008503  0.0001883  -4.516 6.84e-06 ***
FoundationCBlock     0.0583435  0.0105020   5.555 3.30e-08 ***
FoundationPConc      0.0718433  0.0115567   6.217 6.66e-10 ***
FoundationSlab       0.0630207  0.0291017   2.166 0.030512 *  
FoundationStone      0.0038723  0.0390916   0.099 0.921107    
FoundationWood       0.0465776  0.0554929   0.839 0.401418    
Bsmt.QualEx         -0.7570306  0.1136937  -6.659 3.94e-11 ***
Bsmt.QualFa         -0.8357544  0.1124732  -7.431 1.85e-13 ***
Bsmt.QualGd         -0.8407558  0.1123115  -7.486 1.24e-13 ***
Bsmt.QualPo         -0.8411315  0.1443930  -5.825 7.04e-09 ***
Bsmt.QualTA         -0.8503101  0.1122157  -7.577 6.31e-14 ***
Bsmt.ExposureAv      0.1590409  0.0918543   1.731 0.083588 .  
Bsmt.ExposureGd      0.1929930  0.0921545   2.094 0.036415 *  
Bsmt.ExposureMn      0.1151061  0.0920620   1.250 0.211392    
Bsmt.ExposureNo      0.1296663  0.0917945   1.413 0.158000    
log(Total.Bsmt.SF)   0.1191674  0.0091824  12.978  < 2e-16 ***
Heating.QC           0.0091548  0.0033638   2.722 0.006577 ** 
Central.AirY         0.0587416  0.0121548   4.833 1.49e-06 ***
Baths                0.0470497  0.0045251  10.397  < 2e-16 ***
Bedroom.AbvGr       -0.0140992  0.0042615  -3.308 0.000961 ***
Kitchen.AbvGr       -0.0977644  0.0140392  -6.964 5.05e-12 ***
Kitchen.Qual         0.0346078  0.0056027   6.177 8.51e-10 ***
FunctionalMaj2      -0.1661268  0.0518564  -3.204 0.001387 ** 
FunctionalMin1       0.0143539  0.0345741   0.415 0.678085    
FunctionalMin2       0.0352368  0.0342227   1.030 0.303356    
FunctionalMod       -0.0059146  0.0372751  -0.159 0.873948    
FunctionalTyp        0.0882264  0.0312921   2.819 0.004877 ** 
Fireplaces           0.0278606  0.0047452   5.871 5.37e-09 ***
Garage.Cars          0.0400643  0.0046975   8.529  < 2e-16 ***
Paved.DriveP         0.0101521  0.0181899   0.558 0.576854    
Paved.DriveY         0.0573398  0.0114111   5.025 5.67e-07 ***
log(Pool.Area)       0.0175609  0.0059007   2.976 0.002969 ** 
log(TotalSq)         0.3701184  0.0154033  24.029  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.09123 on 1425 degrees of freedom
Multiple R-squared:  0.9426,    Adjusted R-squared:  0.9399 
F-statistic:   349 on 67 and 1425 DF,  p-value: < 2.2e-16
plot(model.BIC)
not plotting observations with leverage one:
  638, 655, 990

not plotting observations with leverage one:
  638, 655, 990

# Exploring the remaining predictors relationship to price
# plot(log(data_train$Lot.Area), log(data_train$price))
# plot((data_train$Neighborhood), log(data_train$price))
# plot((data_train$Condition.1), log(data_train$price))
# plot((data_train$Overall.Qual), log(data_train$price))
# plot((data_train$Overall.Cond), log(data_train$price))
# plot((data_train$HouseAge), log(data_train$price))
# plot((data_train$Bsmt.Qual), log(data_train$price))
# plot((data_train$Bsmt.Exposure), log(data_train$price))
# plot(log(data_train$Total.Bsmt.SF), log(data_train$price))
# plot((data_train$Heating.QC), log(data_train$price))
# plot((data_train$Central.Air), log(data_train$price))
# plot((data_train$Baths), log(data_train$price))
# plot((data_train$Bedroom.AbvGr), log(data_train$price))
# plot((data_train$Kitchen.AbvGr), log(data_train$price))
# plot((data_train$Kitchen.Qual), log(data_train$price))
# plot((data_train$Functional), log(data_train$price))
# plot((data_train$Fireplaces), log(data_train$price))
# plot((data_train$Paved.Drive), log(data_train$price))
# plot((data_train$Garage.Cars), log(data_train$price))
# plot((data_train$Garage.Cars), log(data_train$price))
# plot(log(1+data_train$Pool.Area), log(data_train$price))
# plot(log(data_train$TotalSq), log(data_train$price))
termplot(model.BIC, partial.resid = TRUE, col.res = "purple", cex = 0.5,
         rug = T, se = T, smooth = panel.smooth)

# There are 3 high leverage points - may want to exclude them
hh<-hatvalues(model.BIC)
id<-which(hh==1)
plot(hatvalues(model.BIC), type = "h")

Yhat = predict(model, newdata=data_test, interval="predict")
Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : 
  factor Mas.Vnr.Type has new levels 
sqrt(mean((exp(yhat.ridge)-data_test$price)^2))
[1] 16492.59
bart.gop = bart(x.train= dplyr::select(data_train, -price),
y.train=data_train["price"],
x.test=dplyr::select(data_test, -price),verbose=FALSE)
Error in UseMethod("select_") : 
  no applicable method for 'select_' applied to an object of class "c('double', 'numeric')"

library(MASS)

#Full Model
model=lm(log(price) ~ MS.SubClass + MS.Zoning + log(Lot.Frontage) + log(Lot.Area) +  Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1  + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge + Roof.Style + Roof.Matl + Exterior.1st + Exterior.2nd + Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Qual + Exter.Cond + Foundation + Bsmt.YN:Bsmt.Qual + Bsmt.YN:Bsmt.Cond + Bsmt.YN:Bsmt.Exposure + Total.Bsmt.SF + Heating + Heating.QC + Central.Air + Electrical + log(X12.SF) + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.YN:Fireplace.Qu + Garage.YN:Garage.Type + Garage.YN:Garage.Finish + Garage.Cars + Garage.YN:Garage.Cond + Garage.YN:Garage.Qual + Paved.Drive + log(1+Pool.Area) + Pool.YN:Pool.QC + Fence + Misc.Val + Mo.Sold + Yr.Sold + Sale.Type  + TotalSq, data=train)

summary(model)
boxcox(model)
#BIC model selection
step(model, k=log(nrow(train)))

#step(model, k=2)
# OLD
# model=lm(log(price)~ms.subclass + MS.Zoning + log(Lot.Frontage) + log(area) + Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1 + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + Year.Built + Year.Remod.Add + Roof.Style + Exterior.1st + Exterior.2nd + Mas.Vnr.Type + log(Mas.Vnr.Area + 1) + Exter.Qual + Exter.Cond + Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + Heating.QC + Central.Air + Electrical + log(X1st.Flr.SF) + log(X2nd.Flr.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + TotRms.AbvGrd + Paved.Drive + log(Wood.Deck.SF + 1) + log(Open.Porch.SF + 1) + log(Enclosed.Porch + 1) + log(X3Ssn.Porch + 1) + log(Screen.Porch + 1) + log(Misc.Val + 1) + Mo.Sold + Yr.Sold + Sale.Type + log(TotalSq), data=train)
#vif(model)
#summary(model)

#Model Selection by AIC
#step(model, k=2)
##AIC_model = lm(formula = log(price) ~ PID + area + MS.SubClass + MS.Zoning + 
#    Lot.Area + Lot.Shape + Utilities + Land.Slope + Neighborhood + 
#    Condition.1 + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + 
#    Year.Built + Year.Remod.Add + Roof.Style + Roof.Matl + Exterior.1st + 
#    Exterior.2nd + Exter.Qual + Exter.Cond + Foundation + BsmtFin.SF.1 + 
#    BsmtFin.SF.2 + Bsmt.Unf.SF + Heating + Heating.QC + Central.Air + 
#    Electrical + X1st.Flr.SF + X2nd.Flr.SF + Bsmt.Full.Bath + 
#    Kitchen.Qual + Functional + Garage.Cars + Garage.Area + Paved.Drive + 
#    Wood.Deck.SF + Open.Porch.SF + Enclosed.Porch + Screen.Porch + 
#    Mo.Sold + Yr.Sold + has.fence + has.fireplace, data = train)
#summary(AIC_model)

#Model Selection by BIC
#BIC_model = lm(formula = log(price) ~ PID + area + MS.Zoning + Lot.Area + 
#    Neighborhood + Condition.1 + Bldg.Type + Overall.Qual + Overall.Cond + 
#   Year.Built + Year.Remod.Add + BsmtFin.SF.1 + BsmtFin.SF.2 + 
#    Bsmt.Unf.SF + Central.Air + Bsmt.Full.Bath + Kitchen.Qual + 
#    Functional + Garage.Cars + Garage.Area + Open.Porch.SF + 
#    Enclosed.Porch + Screen.Porch + has.fireplace, data = train)
#summary(BIC_model)

#Use Boosting to find important variables
#
#library(gbm)
#
#boost = gbm(log(price) ~ PID + area + MS.Zoning + Lot.Area + 
#    Neighborhood + Condition.1 + Bldg.Type + Overall.Qual + Overall.Cond + 
#    Year.Built + Year.Remod.Add + BsmtFin.SF.1 + BsmtFin.SF.2 + 
#    Bsmt.Unf.SF + Central.Air + Bsmt.Full.Bath + Kitchen.Qual + 
#    Functional + Garage.Cars + Garage.Area + Open.Porch.SF + 
#    Enclosed.Porch + Screen.Porch + has.fireplace, data = train, distribution="gaussian", n.trees=5000, #interaction.depth = 1, shrinkage=0.01, verbose = F)

#summary(boost)

##Sparse Model (removed variables to bring the number below 20)
#sparse_model = lm(log(price) ~ area + MS.Zoning + Lot.Area + 
#    Neighborhood + Condition.1 + Overall.Qual + Overall.Cond + 
#    Year.Built + Year.Remod.Add + BsmtFin.SF.1 + BsmtFin.SF.2 + 
#    Bsmt.Unf.SF + Central.Air + Bsmt.Full.Bath + Kitchen.Qual + 
#    Functional + Garage.Cars + Garage.Area + has.fireplace, data = train)
#summary(sparse_model)
#plot(sparse_model)
#termplot(sparse_model, 
#    data = train)

##Trying to detect non-linearity
#attach(train)
#plot(log(area), log(price))
#plot(log(Lot.Area), log(price))
#plot(log(Overall.Qual), log(price))
#plot(log(Overall.Cond), log(price))
#plot(log(Year.Built), log(price))
#plot(log(Year.Remod.Add, log(price)))
#plot(BsmtFin.SF.1, log(price))
#plot(BsmtFin.SF.2, log(price))
#plot(Bsmt.Unf.SF, log(price))
#plot(Bsmt.Full.Bath, log(price))
#plot(Garage.Cars, log(price))
#plot(Garage.Area, log(price))
#
##OLD Simple model
#model1 = lm(log(price) ~ log(area) + MS.Zoning + log(Lot.Area) + 
#    Neighborhood + Condition.1 + log(Overall.Qual) + log(Overall.Cond) + 
#    log(Year.Built) + log(Year.Remod.Add) + BsmtFin.SF.1 + BsmtFin.SF.2 + 
#    Bsmt.Unf.SF + Central.Air + Bsmt.Full.Bath +
#    Functional + Garage.Cars + Garage.Area + has.fireplace, data = train[-c(168, 461, 787),])
#summary(model1)
#plot(model1)

#Simple model (based on BIC selection)
model1 = lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF + 
    Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual + 
    Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure, 
    data = train)

#Plot Price Ranges of each neighborhood
library(ggplot2)

neighborhood.price.range = ggplot(train, aes(x = log(price), y = reorder(Neighborhood, desc(log(price)), median))) +
  geom_polygon(color="SlateGray",size = 1, alpha=1) + theme_light() + coord_flip() + theme(axis.text.x=element_text(angle=90,hjust=1,vjust=0.5)) + labs(y="Neighborhood")

plot(neighborhood.price.range)

Model Evaluation on Test Data

Create predicted values for price using your model using the testing data

load("ames_test.Rdata")

#Variables with NA's and their proportion of missing data
miss = apply(is.na(ames_train), 2, sum)
miss_prop = round(miss[miss>0]/nrow(ames_train), 3)
print(miss_prop) 
which(miss_prop>0.5) # four features have greater than 50% of data missing


#Created binary variables for whether or not a house as an alley, pool, fence, misc. feature, fireplace, basement or garage, as I thought it may be more meaningful- Tom
#May consider adding Lot.Frontage back in, as it only had 286 missing
#Have to decide between filtering out N/A's for different Garage variables and only including 'has.garage'
#Have to decide between filtering out N/A's for different Basement variables and only including 'has.basement'
test <- ames_test  %>%   filter(!is.na(Lot.Frontage)) %>% 
  mutate(Alley =  factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
  mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add))  %>%
  filter(!is.na(Mas.Vnr.Area))  %>% 
  mutate(Bsmt.YN = !is.na(Bsmt.Qual)) %>%
    mutate(Bsmt.Qual =  factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "NoBa"), exclude = NULL)) %>%
    mutate(Bsmt.Cond =  factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "NoBa"), exclude = NULL)) %>%
    mutate(Bsmt.Exposure =  factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "NoBa"), exclude = NULL)) %>%
   mutate(BsmtFin.Type.1=  factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "NoBa"), exclude = NULL)) %>%
    mutate(BsmtFin.Type.2=  factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "NoBa"), exclude = NULL)) %>%
  mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF)  %>%   
  filter(!is.na(Bsmt.Full.Bath)) %>%
  filter(!is.na(Bsmt.Half.Bath)) %>%
  mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
  mutate(Fireplace.YN = Fireplaces>0) %>%
  mutate(Fireplace.Qu =  factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
  mutate(Garage.YN = !is.na(Garage.Cond)) %>%
    mutate(Garage.Type =  factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
  mutate(Garage.Finish =  factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
    mutate(Garage.Qual =  factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
      mutate(Garage.Cond =  factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
  mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
  mutate(Pool.YN = Pool.Area>0) %>%
  mutate(Pool.QC =  factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
    mutate(Fence =  factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
  mutate(Misc.Feature =  factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
  mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
    mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
  dplyr::select(-Garage.Yr.Blt) %>% filter(Condition.1 != "RRNe") %>% filter(Kitchen.Qual != "Po")
                  

You should save your predictions in a dataframe with columns for PID (property identifier), fit, predicted values on the test data, and where possible lwr and upr, lower and upper 95% interval estimates for predicting price.

Y = test$price
Yhat = exp(Yhat)

#Bias
mean(Yhat[,1] - Y)

#Maximum Deviation
max(abs(Y - Yhat[,1]))

#Mean Absolute Deviation
mean(abs(Y - Yhat[,1]))

#RMSE
sqrt(mean((Y - Yhat[,1])^2))

#Coverage
mean(Yhat[,"lwr"] < Y & Yhat[,"upr"] > Y)

# name dataframe as predictions! DO NOT CHANGE
predictions = as.data.frame(Yhat)
predictions$PID = test$PID
save(predictions, file="predict.Rdata")

Your models will be evaluated on the following criteria on the test data:

* Bias: Average (Yhat-Y) positive values indicate the model tends to overestimate price (on average) while negative values indicate the model tends to underestimate price.

* Maximum Deviation: Max |Y-Yhat| - identifies the worst prediction made in the validation data set.

* Mean Absolute Deviation: Average |Y-Yhat| - the average error (regardless of sign).

  • Root Mean Square Error: Sqrt Average (Y-Yhat)^2

  • Coverage: Average( lwr < Y < upr)

In order to have a passing wercker badge, your file for predictions needs to be the same length as the test data, with three columns: fitted values, lower CI and upper CI values in that order with names, fit, lwr, and upr respectively.

You will be able to see your scores on the score board (coming soon!). They will be initialized by a predction based on the mean in the training data.

Model Check - Test your prediction on the first observation in the training and test data set to make sure that the model gives a reasonable answer and include this in a supplement of your report. This should be done BY HAND using a calculator (this means use the raw data from the original dataset and manually calculate all transformations and interactions with your calculator)! Models that do not give reasonable answers will be given a minimum 2 letter grade reduction. Also be careful as you cannot use certain transformations [log or inverse x] if a variable has values of 0.

Part II: Complex Model

In this part you may go all out for constructing a best fitting model for predicting housing prices using methods that we have covered this semester. You should feel free to to create any new variables (such as quadratic, interaction, or indicator variables, splines, etc). The variable TotalSq = X1st.Flr.SF+X2nd.Flr.SF was added to the dataframe (that does not include basement area, so you may improve on this. A relative grade is assigned by comparing your fit on the test set to that of your fellow students with bonus points awarded to those who substantially exceed their fellow students and point reductions occurring for models which fit exceedingly poorly.

library(glmnet)
#Design matrices
X.train = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF + 
    Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual + 
    Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure, 
    data = data)

X.test = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF + 
    Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual + 
    Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure, 
    data = test)

#Fit lasso regression
price.lasso = glmnet(X.train, log(data$price), alpha=1)

cv.lasso = cv.glmnet(X.train, log(data$price), alpha=1)

#Obtain prediction on test data
yhat.lasso.test = predict(app.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test)

#Compute RMSE for test data
rmse.lasso.test = rmse(test$price, exp(yhat.lasso.test))
rmse.tab$lasso<-rmse.lasso.test

Update your predictions using your complex model to provide point estimates and CI.

You may iterate here as much as you like exploring different models until you are satisfied with your results.

Part III: Write Up

Once you are satisfied with your model, provide a write up of your data analysis project in a new Rmd file/pdf file: writeup.Rmd by copying over salient parts of your R notebook. The written assignment consists of five parts:

  1. Exploratory data analysis (20 points): must include three correctly labeled graphs and an explanation that highlight the most important features that went into your model building.

  2. Development and assessment of an initial model from Part I (10 points)

  • Initial model: must include a summary table and an explanation/discussion for variable selection. Interpretation of coefficients desirable for full points.

  • Model selection: must include a discussion

  • Residual: must include a residual plot and a discussion

  • RMSE: must include an RMSE and an explanation (other criteria desirable)

  • Model testing: must include an explanation

  1. Development of the final model (20 points)
  • Final model: must include a summary table

  • Variables: must include an explanation

  • Variable selection/shrinkage: must use appropriate method and include an explanation

  1. Assessment of the final model (25 points)
  • Residual: must include a residual plot and a discussion

  • RMSE: must include an RMSE and an explanation (other criteria desirable)

  • Model evaluation: must include an evaluation discussion

  • Model testing : must include a discussion

  • Model result: must include a selection of the top 10 undervalued and overvalued houses

  1. Conclusion (10 points): must include a summary of results and a discussion of things learned

Part IV

Create predictions for the validation data from your final model and write out to a file prediction-validation.Rdata This should have the same format as the models in Part I and II.

10 points

Class Presentations

Each Group should prepare 5 slides in their Github repo: (save as slides.pdf)

  • Most interesting graphic (a picture is worth a thousand words prize!)

  • Best Model (motivation, how you found it, why you think it is best)

  • Best Insights into predicting Sales Price.

  • 2 Best Houses to purchase (and why)

  • Best Team Name/Graphic

We will select winners based on the above criteria and overall performance.

Finally your repo should have: writeup.Rmd, writeup.pdf, slides.Rmd (and whatever output you use for the presentation) and predict.Rdata and predict-validation.Rdata.

LS0tCnRpdGxlOiAiRmluYWwgRGF0YSBBbmFseXNpcyBQcm9qZWN0IgpkYXRlOiAgIldyaXRlIHVwIGR1ZSBBcHJpbCAyOCBhdCA1IHBtIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFKQpgYGAKCgpGb3IgdGhpcyBwcm9qZWN0IHlvdSB3aWxsIHRha2UgdGhlIHJvbGUgb2YgYSBjb25zdWx0YW50IGhpcmVkIGJ5IGEgcmVhbCBlc3RhdGUgaW52ZXN0bWVudCBmaXJtIGluIEFtZXMsIElvd2EsIGEgbWlkLXdlc3QgdG93biBpbiB0aGUgVW5pdGVkIFN0YXRlcywgdG8gYW5hbHl6ZSBkYXRhIGluIG9yZGVyIHRvIGhlbHAgcHJvdmlkZSBpbnNpZ2h0IGludG8gaG93IHRoZSBmaXJtIHNob3VsZCBpbnZlc3QgZm9yIGhpZ2hlc3QgcHJvZml0cywgYW5kIHRvIHF1YW50aWZ5IGFuZCBjb21tdW5pY2F0ZSB0byB0aGUgY29tcGFueSBtYW5hZ2VtZW50IHdoYXQgdHlwZXMgb2YgcmVhbCBlc3RhdGUgcHJvcGVydGllcyBhcmUgZ29vZCBpbnZlc3RtZW50cyBhbmQgd2h5LiBUaGV5IGhhdmUgcHJvdmlkZWQgeW91IHdpdGggZGF0YSBvbiBob3VzaW5nIHNhbGVzIGZyb20gYmV0d2VlbiAyMDA2IHRvIDIwMTAgdGhhdCBjb250YWlucyBpbmZvcm1hdGlvbiBhYm91dCB0aGUgY2hhcmFjdGVyaXN0aWNzIG9mIHRoZSBob3VzZSAobnVtYmVyIG9mIGJlZHJvb21zLCBudW1iZXIgb2YgYmF0aHJvb21zLCBzcXVhcmUgZm9vdGFnZSwgZXRjLikgYW5kIHRoZSBob3VzZSdzIHNhbGUgcHJpY2UuIFRoZSBjb2RlYm9vayBmb3IgdGhpcyBkYXRhIHNldCBpcyBbYXZhaWxhYmxlIG9ubGluZSBoZXJlIF0oaHR0cHM6Ly93dzIuYW1zdGF0Lm9yZy9wdWJsaWNhdGlvbnMvanNlL3YxOW4zL2RlY29jay9kYXRhZG9jdW1lbnRhdGlvbi50eHQpICBvciBpbiB0aGUgRGF0YSBmb2xkZXIgaW4geW91ciByZXBvLgoKIyMgQWJvdXQgdGhlIERhdGEgQW5hbHlzaXMgUHJvamVjdAoKSXQncyBnZW5lcmFsbHkgYSBiYWQgaWRlYSB0byBidXkgdGhlIG1vc3QgZXhwZW5zaXZlIGhvdXNlIGluIHRoZSBuZWlnaGJvcmhvb2QuIEFuZCByZW1lbWJlciB0aGUgcmVhbCBlc3RhdGUgYWdlbnRzJyBtYW50cmE6IExvY2F0aW9uLCBsb2NhdGlvbiwgbG9jYXRpb24hIEtlZXAgaW4gbWluZCB0aGF0IHRoZSBnb2FsIGlzIHRvIG1ha2UgbW9uZXkgZm9yIHlvdXIgaW52ZXN0b3JzLCBhbmQgaGVuY2UgaW52ZXN0aW5nIGluIGEgcHJvcGVydHkgdGhhdCBpcyBvdmVydmFsdWVkIChjb3N0aW5nIG1vcmUgdGhhbiBpdCBpcyB3b3J0aCkgaXMgcmFyZWx5IGEgZ29vZCBpZGVhLiBUaGlzIG1lYW5zIHRoYXQgaXQncyBjcml0aWNhbCB0byBrbm93IHdoaWNoIHByb3BlcnRpZXMgYXJlIG92ZXJ2YWx1ZWQgYW5kIHdoaWNoIGFyZSB1bmRlcnZhbHVlZC4gIFRoZSBjb21wYW55IHRoYXQgaGlyZWQgeW91IGhhcyBtYW55IHF1ZXN0aW9ucyBmb3IgeW91IGFib3V0IHRoZSBob3VzaW5nIG1hcmtldC4gSXQgaXMgdXAgdG8geW91IHRvIGRlY2lkZSB3aGF0IG1ldGhvZHMgeW91IHdhbnQgdG8gdXNlIChmcmVxdWVudGlzdCBvciBCYXllc2lhbikgdG8gYW5zd2VyIHRoZXNlIHF1ZXN0aW9ucywgYW5kIGltcGxlbWVudCB0aGVtIHRvIGhlbHAgdG8gaWRlbnRpZnkgdW5kZXJ2YWx1ZWQgYW5kIG92ZXJ2YWx1ZWQgcHJvcGVydGllcy4KCgpZb3Ugd2lsbCBoYXZlIHRocmVlIGRhdGEgc2V0czogYSBzdWJzZXQgZm9yIHRyYWluaW5nLCBhIHN1YnNldCBmb3IgdGVzdGluZywgYW5kIGEgdGhpcmQgc3Vic2V0IGZvciB2YWxpZGF0aW9uLiBZb3Ugd2lsbCBiZSBhc2tlZCB0byBkbyBkYXRhIGV4cGxvcmF0aW9uIGFuZCBidWlsZCB5b3VyIG1vZGVsIChvciBtb2RlbHMpIGluaXRpYWxseSB1c2luZyBvbmx5IHRoZSB0cmFpbmluZyBkYXRhLiBUaGVuLCB5b3Ugd2lsbCB0ZXN0IHlvdXIgbW9kZWwgb24gdGhlIHRlc3RpbmcgZGF0YSwgYW5kIGZpbmFsbHkgdmFsaWRhdGUgdXNpbmcgdGhlIHZhbGlkYXRpb24gZGF0YS4gV2UgYXJlIGNoYWxsZW5naW5nIHlvdSB0byBrZWVwIHlvdXIgYW5hbHlzaXMgZXhwZXJpZW5jZSByZWFsaXN0aWMsIGFuZCBpbiBhIHJlYWxpc3RpYyBzY2VuYXJpbyB5b3Ugd291bGQgbm90IGhhdmUgYWNjZXNzIHRvIGFsbCB0aHJlZSBvZiB0aGVzZSBkYXRhIHNldHMgYXQgb25jZS4gIFlvdSB3aWxsIGJlIGFibGUgdG8gc2VlIG9uIG91ciBzY29yZWJvYXJkIGhvdyB3ZWxsIHlvdXIgdGVhbSBpcyBkb2luZyBiYXNlZCBvbiBpdHMgcHJlZGljdGl2ZSBwZXJmb3JtYW5jZSBvbiB0aGUgdGVzdGluZyBkYXRhLiAgQWZ0ZXIgeW91ciBwcm9qZWN0IGlzIHR1cm5lZCBpbiB5b3Ugd2lsbCBzZWUgdGhlIGZpbmFsIHNjb3JlIG9uIHRoZSB2YWxpZGF0aW9uIHNldC4KCkFsbCBtZW1iZXJzIG9mIHRoZSB0ZWFtIHNob3VsZCBjb250cmlidXRlIGVxdWFsbHkgYW5kIGFuc3dlciBhbnkgcXVlc3Rpb25zIGFib3V0IHRoZSBhbmFseXNpcyBhdCB0aGUgZmluYWwgcHJlc2VudGF0aW9uLgoKRm9yIHlvdXIgYW5hbHlzaXMgY3JlYXRlIGEgbmV3IG5vdGVib29rIG5hbWVkICJwcm9qZWN0LlJtZCIKYW5kIHVwZGF0ZSBhY2NvcmRpbmdseSByYXRoZXIgdGhhbiBlZGl0aW5nIHRoaXMuCgoKIyMjIFJlYWQgaW4gVHJhaW5pbmcgRGF0YQoKVG8gZ2V0IHN0YXJ0ZWQgcmVhZCBpbiB0aGUgdHJhaW5pbmcgZGF0YS4gCmBgYHtyIHJlYWQtZGF0YX0KbGlicmFyeShkcGx5cikKbGlicmFyeSh0aWR5cikKbG9hZCgiYW1lc190cmFpbi5SZGF0YSIpCgpwcmludChwYXN0ZTAoIlRoZSBkYXRhc2V0IGhhcyAiLCBkaW0oYW1lc190cmFpbilbMV0sICIgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBhbmQgIiwgZGltKGFtZXNfdHJhaW4pWzJdLCAiIGZlYXR1cmVzIikpCgojVmFyaWFibGVzIHdpdGggTkEncyBhbmQgdGhlaXIgcHJvcG9ydGlvbiBvZiBtaXNzaW5nIGRhdGEKbWlzcyA9IGFwcGx5KGlzLm5hKGFtZXNfdHJhaW4pLCAyLCBzdW0pCm1pc3NfcHJvcCA9IHJvdW5kKG1pc3NbbWlzcz4wXS9ucm93KGFtZXNfdHJhaW4pLCAzKQpwcmludChtaXNzX3Byb3ApIAp3aGljaChtaXNzX3Byb3A+MC41KSAjIGZvdXIgZmVhdHVyZXMgaGF2ZSBncmVhdGVyIHRoYW4gNTAlIG9mIGRhdGEgIm1pc3NpbmciIC0tIGRyb3AgdGhlc2UgdmFyaWFibGVzCmBgYAoKCk5vdGVzIGFib3V0IGRhdGEgY2xlYW5pbmc6CgpXZSBkcm9wcGVkIOKAnHV0aWxpdGllc+KAnSAodHlwZSBvZiB1dGlsaXRpZXMgYXZhaWxhYmxlKSBzaW5jZSBpbiB0aGUgdHJhaW5pbmcgc2V0LCBvbmx5IDIgb2JzZXJ2YXRpb25zIGRpZCBub3QgaGF2ZSBhbGwgdGhlIHV0aWxpdGllcyAoZWxlY3RyaWNpdHksIGdhcywgd2F0ZXIgYW5kIHNld2FnZSkuIEludHVpdGl2ZWx5LCBtb3N0IG1vZGVybiBwcm9wZXJ0eSBhcmUgZXF1aXBwZWQgd2l0aCB0aGVzZSBiYXNpYyBwdWJsaWMgdXRpbGl0aWVzIGFuZCBrZWVwaW5nIHRoZSB2YXJpYWJsZSB3b3VsZCB0aGVyZWZvcmUgYmUgdW5uZWNlc3NhcnkuIAoKV2UgYWxzbyBkcm9wcGVkIOKAnGNvbmRpdGlvbiAy4oCdIChwcm94aW1pdHkgdG8gdmFyaW91cyBjb25kaXRpb25zIGlmIG1vcmUgdGhhbiBvbmUgaXMgcHJlc2VudCkgc2luY2UgaXQgc2VlbWVkIHJlZHVuZGFudCBmcm9tIG91ciB0cmFpbmluZyBzZXQuIEdpdmVuIENvbmRpdGlvbiAxLCBvbmx5IDEyIHByb3BlcnRpZXMgd2VyZSBub3QgY2xvc2UgdG8gbm9ybWFsIGNvbmRpdGlvbnMuIAoKSW4gdGhlIG9yaWdpbmFsIHNjYWxlLCAxOTkwIGFuZCAxOTAwIHdvdWxkIG5vdCBiZSBtdWNoIGRpZmZlcmVudC4gVGhlcmVmb3JlLCB3ZSBjaGFuZ2VkIHRoZSBzY2FsZSB0byB0aGUgbnVtYmVyIG9mIHllYXJzIHNpbmNlIGxhc3QgY29uc3RydWN0aW9uIG9yIHJlbW9kZWxsaW5nLCBzdWJ0cmFjdGVkIGZyb20geWVhciAyMDEwICh0aGUgZW5kIHllYXIgaW4gdGhlIGRhdGFzZXQpLiAKCkFub3RoZXIgdmFyaWFibGUgZHJvcHBlZCBpbiBvdXIgbW9kZWwgd2FzIOKAnHJvb2YgbWF0ZXJpYWzigJ0gc2luY2Ugb25seSAxJSBvZiB0aGUgcHJvcGVydHkgdXNlZCBtYXRlcmlhbCBvdGhlciB0aGFuIOKAnHN0YW5kYXJkIGNvbXBvc2l0ZSBzaGluZ2xl4oCdLiBTaW1pbGFybHksIOKAnGhlYXRpbmfigJ0gd2FzIGFsc28gZHJvcHBlZCBzaW5jZSBtb3JlIHRoYW4gOTUlIG9mIHRoZSBwcm9wZXJ0eSBoYXMgZ2FzIGZvcmNlZCB3YXJtIGFpciBmdXJuYWNlIChHYXNBKSBpbnN0ZWFkIG9mIG90aGVyIHR5cGVzIG9mIGhlYXRpbmcuCgpGb3Ig4oCcZXh0ZXJpb3IgcXVhbGl0eeKAnSBhbmQg4oCcZXh0ZXJpb3IgY29uZGl0aW9u4oCdCSwgd2UgcmVjb2RlZCB0aGVzZSBvcmRpbmFsIHZhcmlhYmxlcyB0byAxLTUgdG8gcmVwbGFjZSB0aGUgb3JpZ2luYWwgc2NhbGUgb2YgY29uZGl0aW9ucyAoZnJvbSBwb29yIHRvIGV4Y2VsbGVudCkuIFNpbWlsYXJseSwgd2UgcmVjb2RlZCDigJxiYXNlbWVudCBleHBvc3VyZeKAnSBhbmQg4oCcYmFzZW1lbnQgcmF0aW5n4oCdIGV4Y2VwdCB0aGF0IHRoZSBuZXcgc2NhbGUgd291bGQgc3RhcnQgZnJvbSAwIGZvciBwcm9wZXJ0aWVzIHdpdGhvdXQgYmFzZW1lbnQuIAoJCQkJCQkJCQkJCQkKRXh0ZXIgUXVhbCAoT3JkaW5hbCk6IEV2YWx1YXRlcyB0aGUgcXVhbGl0eSBvZiB0aGUgbWF0ZXJpYWwgb24gdGhlIGV4dGVyaW9yCgkJCQkJCQogICAgICBFeCAgICAgICBFeGNlbGxlbnQKICAgICAgIEdkICAgICAgIEdvb2QKICAgICAgIFRBICAgICAgIEF2ZXJhZ2UvVHlwaWNhbAogICAgICAgRmEgICAgICAgRmFpcgogICAgICAgUG8gICAgICAgUG9vcgoKCgkJCQkJCQpFeHRlciBDb25kIChPcmRpbmFsKTogRXZhbHVhdGVzIHRoZSBwcmVzZW50IGNvbmRpdGlvbiBvZiB0aGUgbWF0ZXJpYWwgb24gdGhlIGV4dGVyaW9yCgogICAgICBFeCAgICAgICBFeGNlbGxlbnQKICAgICAgIEdkICAgICAgIEdvb2QKICAgICAgIFRBICAgICAgIEF2ZXJhZ2UvVHlwaWNhbAogICAgICAgRmEgICAgICAgRmFpcgogICAgICAgUG8gICAgICAgUG9vcgoKCkNvbnRpbnVvdXMgdmFyaWFibGVzIHN1Y2ggYXMg4oCcMXN0IGZsb29yIHNxdWFyZSBmZWV04oCdIGFuZCDigJwybmQgZmxvb3Igc3F1YXJlIGZlZXTigJ0gd2VyZSBsb2ctdHJhbnNmb3JtZWQgZm9yIGludGVycHJldGF0aW9uIHB1cnBvc2UuIAoKRm9yIHZhcmlhYmxlIOKAnGZ1bmN0aW9uYWzigJ0sIHdlIHJlY29kZWQgZGlmZmVyZW50IG9yZGluYWwgbGV2ZWxzIGludG8gYmluYXJ5IGxldmVscyAtLS0gdHlwaWNhbCBmdW5jdGlvbmFsaXR5IG9yIG5vdCwgaW5jbHVkaW5nIG1pbm9yIGFuZCBtYWpvciBkZWR1Y3Rpb25zLiAgCQkKCldlIHN1bW1lZCB1cCB0aGUgbnVtYmVyIG9mIGJhdGhyb29tcyB0byBvbmUgY29udGludW91cyB2YXJpYWJsZS4gTm90ZSB0aGF0IG9uZSBoYWxmLWJhdGhyb29tIHdvdWxkIGJlIGNvZGVkIGFzIDAuNS4JCmBgYHtyfQojIERpZCBub3QgcmVtb3ZlIGFueSBOQSBlbnRyaWVzIGluIExvdC5mcm9udGFnZQoKZGF0YT1hbWVzX3RyYWluCgpkYXRhIDwtIGRhdGEgICU+JQogICNmaWx0ZXIoIWlzLm5hKExvdC5Gcm9udGFnZSkpICU+JSAKICBtdXRhdGUoTVMuU3ViQ2xhc3M9IGZhY3RvcihNUy5TdWJDbGFzcykpICU+JQogIG11dGF0ZShBbGxleSA9ICBmYWN0b3IoQWxsZXksIGxldmVscyA9IGxldmVscyhhZGROQShBbGxleSkpLCBsYWJlbHMgPSBjKGxldmVscyhBbGxleSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEhvdXNlQWdlID0gWXIuU29sZC0gcG1heChZZWFyLkJ1aWx0LCBZZWFyLlJlbW9kLkFkZCkpICAlPiUKICBmaWx0ZXIoIWlzLm5hKE1hcy5WbnIuQXJlYSkpICAlPiUgCiAgbXV0YXRlKEJzbXQuWU4gPSAxKighaXMubmEoQnNtdC5RdWFsKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSAgZmFjdG9yKEJzbXQuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LlF1YWwpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSByZWxldmVsKEJzbXQuUXVhbCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210LkNvbmQgPSAgZmFjdG9yKEJzbXQuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuQ29uZCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LkNvbmQpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LkNvbmQgPSByZWxldmVsKEJzbXQuQ29uZCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210LkV4cG9zdXJlID0gIGZhY3RvcihCc210LkV4cG9zdXJlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5FeHBvc3VyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LkV4cG9zdXJlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9IHJlbGV2ZWwoQnNtdC5FeHBvc3VyZSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMT0gIGZhY3RvcihCc210RmluLlR5cGUuMSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4xKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4xKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjEgPSByZWxldmVsKEJzbXRGaW4uVHlwZS4xLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4yPSAgZmFjdG9yKEJzbXRGaW4uVHlwZS4yLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdEZpbi5UeXBlLjIpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdEZpbi5UeXBlLjIpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMiA9IHJlbGV2ZWwoQnNtdEZpbi5UeXBlLjIsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoWDEyLlNGPSBYMXN0LkZsci5TRisgWDJuZC5GbHIuU0YpICAlPiUgICAKICBmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IDEqKEZpcmVwbGFjZXM+MCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSAgZmFjdG9yKEZpcmVwbGFjZS5RdSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEZpcmVwbGFjZS5RdSkpLCBsYWJlbHMgPSBjKGxldmVscyhGaXJlcGxhY2UuUXUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSByZWxldmVsKEZpcmVwbGFjZS5RdSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuWU4gPSAxKighaXMubmEoR2FyYWdlLkNvbmQpKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlR5cGUgPSByZWxldmVsKEdhcmFnZS5UeXBlLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5GaW5pc2ggPSAgZmFjdG9yKEdhcmFnZS5GaW5pc2gsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuRmluaXNoKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5GaW5pc2gpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShHYXJhZ2UuRmluaXNoID0gcmVsZXZlbChHYXJhZ2UuRmluaXNoLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5RdWFsID0gIGZhY3RvcihHYXJhZ2UuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5RdWFsKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5RdWFsKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlF1YWwgPSByZWxldmVsKEdhcmFnZS5RdWFsLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5Db25kID0gIGZhY3RvcihHYXJhZ2UuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5Db25kKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkNvbmQgPSByZWxldmVsKEdhcmFnZS5Db25kLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKFBvcmNoLkFyZWEgPSBXb29kLkRlY2suU0YrIE9wZW4uUG9yY2guU0YrRW5jbG9zZWQuUG9yY2grWDNTc24uUG9yY2ggKyBTY3JlZW4uUG9yY2gpICU+JQogIG11dGF0ZShQb29sLllOID0gMSooUG9vbC5BcmVhPjApKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKFBvb2wuUUMgPSByZWxldmVsKFBvb2wuUUMsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICBtdXRhdGUoWXIuU29sZCA9IGFzLmZhY3RvcihZci5Tb2xkKSkgJT4lCiAgZHBseXI6OnNlbGVjdCgtR2FyYWdlLllyLkJsdCkgJT4lCiAgbXV0YXRlKENvbmRpdGlvbi4xID0gYXMuY2hhcmFjdGVyKENvbmRpdGlvbi4xKSkgJT4lCiAgbXV0YXRlKEtpdGNoZW4uUXVhbD1wbHlyOjptYXB2YWx1ZXMoS2l0Y2hlbi5RdWFsLCBmcm9tID0gYygiUG8iLCAiRmEiLCAiVEEiLCJHZCIsICJFeCIgKSwgdG8gPSBjKCIxIiwgIjIiLCAiMyIsICI0IiwgIjUiKSkpICU+JQogIG11dGF0ZShLaXRjaGVuLlF1YWwgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihLaXRjaGVuLlF1YWwpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUM9cGx5cjo6bWFwdmFsdWVzKEhlYXRpbmcuUUMsIGZyb20gPSBjKCJQbyIsICJGYSIsICJUQSIsIkdkIiwgIkV4IiApLCB0byA9IGMoIjEiLCAiMiIsICIzIiwgIjQiLCAiNSIpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUMgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihIZWF0aW5nLlFDKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSBkcm9wbGV2ZWxzKEJzbXQuUXVhbCkpICU+JQogIG11dGF0ZShGdW5jdGlvbmFsID0gZHJvcGxldmVscyhGdW5jdGlvbmFsKSkgJT4lCiAgbXV0YXRlKFJvb2YuTWF0bCA9IGRyb3BsZXZlbHMoUm9vZi5NYXRsKSkKCgoKIyBTaW1wbGlmeSBDb25kaXRpb24gMSAoUGFyaywgUmFpbCwgTm9ybWFsKQppbmRfcmFpbDwtd2hpY2goZGF0YSRDb25kaXRpb24uMT09IlJSTm4iIHwgZGF0YSRDb25kaXRpb24uMT09IlJSQW4iIHwgZGF0YSRDb25kaXRpb24uMT09IlJSTmUiIHwgZGF0YSRDb25kaXRpb24uMT09IlJSQWUiKQppbmRfcGFyazwtd2hpY2goZGF0YSRDb25kaXRpb24uMT09IlBvc04iIHwgZGF0YSRDb25kaXRpb24uMT09IlBvc0EiKQpkYXRhJENvbmRpdGlvbi4xW2luZF9yYWlsXTwtIlJhaWwiCmRhdGEkQ29uZGl0aW9uLjFbaW5kX3BhcmtdPC0iUGFyayIKZGF0YSA9IGRhdGEgJT4lIAogIG11dGF0ZShDb25kaXRpb24uMSA9IGZhY3RvcihDb25kaXRpb24uMSkpICU+JQogIG11dGF0ZShDb25kaXRpb24uMSA9IHJlbGV2ZWwoQ29uZGl0aW9uLjEsIHJlZj0iTm9ybSIpKSAKCiMgRWxpbWluYXRlIHRoZSBvbmUgZW50cnkgaW4gJ0V4cG9zdXJlJyB0aGF0IGhhZCBiZWVuIGxlZnQgY29tcGxldGVseSBlbXB0eQpkYXRhX3RyYWluPC1kYXRhCmRhdGFfdHJhaW4kQnNtdC5FeHBvc3VyZVt3aGljaChkYXRhX3RyYWluJEJzbXQuRXhwb3N1cmU9PSIiKV08LSJOb25lIgpkYXRhX3RyYWluJEJzbXQuRXhwb3N1cmU8LWRyb3BsZXZlbHMoZGF0YV90cmFpbiRCc210LkV4cG9zdXJlKQoKZGF0YV90cmFpbiRQb29sLkFyZWE8LWRhdGFfdHJhaW4kUG9vbC5BcmVhKzEKZGF0YV90cmFpbiRUb3RhbC5Cc210LlNGPC1kYXRhX3RyYWluJFRvdGFsLkJzbXQuU0YrMQpgYGAKClRoZSBgTmVpZ2hib3Job29kYCB2YXJpYWJsZSwgdHlwaWNhbGx5IG9mIGxpdHRsZSBpbnRlcmVzdCBvdGhlciB0aGFuIHRvIG1vZGVsIHRoZSBsb2NhdGlvbiBlZmZlY3QsIG1heSBiZSBvZiBtb3JlIHJlbGV2YW5jZSB3aGVuIHVzZWQgd2l0aCB0aGUgW21hcF0oaHR0cDovL3d3dy5hbXN0YXQub3JnL3B1YmxpY2F0aW9ucy9qc2UvdjE5bjMvZGVjb2NrL0FtZXNSZXNpZGVudGlhbC5wZGYpLgoKV2UgYXJlIHJlc3RyaWN0aW5nIGF0dGVudGlvbiB0byBqdXN0IHRoZSAibm9ybWFsIHNhbGVzIiBjb25kaXRpb24uCgoKIyMgUGFydCBJOiBTaW1wbGUgTW9kZWwKCkluIHRoZSBmaXJzdCBtb2RlbCB5b3UgYXJlIGFsbG93ZWQgb25seSBsaW1pdGVkIG1hbmlwdWxhdGlvbnMgb2YgdGhlIG9yaWdpbmFsIGRhdGEgc2V0IHRvIHByZWRpY3QgdGhlIHNhbGVzIHByaWNlIGBwcmljZWAuIFlvdSBhcmUgYWxsb3dlZCB0byB0YWtlIHBvd2VyIHRyYW5zZm9ybWF0aW9ucyBvZiB0aGUgb3JpZ2luYWwgdmFyaWFibGVzIFtzcXVhcmUgcm9vdHMsIGxvZ3MsIGludmVyc2VzLCBzcXVhcmVzLCBldGMuXSBidXQgeW91IGFyZSBOT1QgYWxsb3dlZCB0byBjcmVhdGUgaW50ZXJhY3Rpb24gdmFyaWFibGVzLiBUaGlzIG1lYW5zIHRoYXQgYSB2YXJpYWJsZSBtYXkgb25seSBiZSB1c2VkIG9uY2UgaW4gYW4gZXF1YXRpb24gW2lmIHlvdSB1c2UgJCB4XjIkIGRvbuKAmXQgdXNlICR4JF0uIEFkZGl0aW9uYWxseSwgeW91IG1heSBlbGltaW5hdGUgYW55IGRhdGEgcG9pbnRzIHlvdSBkZWVtIHVuZml0LiBUaGlzIG1vZGVsIHNob3VsZCBoYXZlIGEgbWluaW11bSByLXNxdWFyZSBvZiA3MyUgKGluIHRoZSBvcmlnaW5hbCB1bml0cykgYW5kIGNvbnRhaW4gYXQgbGVhc3QgNiB2YXJpYWJsZXMgYnV0IGZld2VyIHRoYW4gMjAuICAgCgoKCgpgYGB7cn0KIyMjIHBlcmZyb21hbmNlIGV2bGF1dGlvbiBmdW5jdGlvbgoKCnBlcmZvcm1hbmNlPC0gZnVuY3Rpb24oWSwgWWhhdCl7CiAgCiAgYmlhczwtIG1lYW4oWS1ZaGF0WywxXSkKICAKICBtYXguZGV2PC1tYXgoYWJzKFktWWhhdFssMV0pKQogIAogIG1lYW4uZGV2PC1tZWFuKGFicyhZLVloYXRbLDFdKSkKICAKICBSTVNFPC1zcXJ0KG1lYW4oKFktWWhhdFssMV0pXjIpKQogIAogIAogIGNvdmVyYWdlPC1tZWFuKChZPlloYXRbLDJdKSAmIChZPFloYXRbLDNdKSkKICAKICBvdXQ8LWRhdGEuZnJhbWUoYmlhcz1iaWFzLCBtYXguZGV2PW1heC5kZXYsIG1lYW4uZGV2PW1lYW4uZGV2LCBSTUVTPVJNU0UsIENvdmVyYWdlPWNvdmVyYWdlKQogIAogIHJldHVybihvdXQpCn0KCmBgYAoKCgoKCmBgYHtyfQpsaWJyYXJ5KE1BU1MpCgojIEJhc2UgbW9kZWwgd2l0aCB0cmFuc2Zvcm1lZCBwcmVkaWN0b3JzIAogbW9kZWw9bG0ocHJpY2UgfiBNUy5TdWJDbGFzcyArIE1TLlpvbmluZyArIGxvZyhMb3QuRnJvbnRhZ2UpICsgbG9nKExvdC5BcmVhKSArICBTdHJlZXQgKyBBbGxleSArIExvdC5TaGFwZSArIExhbmQuQ29udG91ciArIExvdC5Db25maWcgKyBMYW5kLlNsb3BlICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgICsgQmxkZy5UeXBlICsgSG91c2UuU3R5bGUgKyBPdmVyYWxsLlF1YWwgKyBPdmVyYWxsLkNvbmQgKyBIb3VzZUFnZSArICAgUm9vZi5TdHlsZSArIFJvb2YuTWF0bCArIEV4dGVyaW9yLjFzdCArIE1hcy5WbnIuVHlwZSArIGxvZygxK01hcy5WbnIuQXJlYSkgKyBFeHRlci5Db25kICsgRXh0ZXIuUXVhbCArICAgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBUb3RhbC5Cc210LlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEVsZWN0cmljYWwgKyBsb2coWDEyLlNGKSArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuUXUgKyBHYXJhZ2UuVHlwZSArIEdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5Db25kICsgR2FyYWdlLlF1YWwgKyBQYXZlZC5Ecml2ZSArIGxvZygxK1Bvb2wuQXJlYSkgKyBQb29sLlFDICsgRmVuY2UgKyBNaXNjLlZhbCArIE1vLlNvbGQgICtZci5Tb2xkICsgU2FsZS5UeXBlICArIFRvdGFsU3EsIGRhdGE9ZGF0YV90cmFpbikKCgojIEJveGNveCAoaW5kaWNhdGVzIHRoYXQgbG9nIGlzIGRlY2VudCkKbDwtYm94Y294KG1vZGVsKQpleHBvPC1yb3VuZChsJHhbd2hpY2gubWF4KGwkeSldLDIpCgojIyBDdXJyZW50IG1vZGVsCiMjIGxvZyhMb3QuRnJvbnRhZ2UpIGN1cnJlbnRseSByZW1vdmVkIHRvIGhhdmUgbW9yZSBkYXRhIHBvaW50cyAod2FzIG5vdCBpbmNsdWRlZCB3aGVuIGxlZnQgaW4gdGhlIG1vZGVsIHdpdGggQklDKQptb2RlbC4wPWxtKGxvZyhwcmljZSkgfiBNUy5TdWJDbGFzcyArIE1TLlpvbmluZyAgKyBsb2coTG90LkFyZWEpICsgIFN0cmVldCArIEFsbGV5ICsgTG90LlNoYXBlICsgTGFuZC5Db250b3VyICsgTG90LkNvbmZpZyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSAgKyBCbGRnLlR5cGUgKyBIb3VzZS5TdHlsZSArIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgICBSb29mLlN0eWxlICsgUm9vZi5NYXRsICsgRXh0ZXJpb3IuMXN0ICsgIE1hcy5WbnIuVHlwZSArIGxvZygxK01hcy5WbnIuQXJlYSkgKyBFeHRlci5Db25kICsgRXh0ZXIuUXVhbCArICAgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBCc210LllOKyBIZWF0aW5nICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuUXUgKyBHYXJhZ2UuVHlwZSArIEdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArR2FyYWdlLlF1YWwrR2FyYWdlLkNvbmQgICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIFBvb2wuUUMgKyBGZW5jZSArIE1pc2MuVmFsICsgTW8uU29sZCAgK1lyLlNvbGQgKyBTYWxlLlR5cGUgICsgbG9nKFRvdGFsU3EpICsgUG9vbC5ZTiAsIGRhdGE9ZGF0YV90cmFpbikKCgojIFRoZXJlIGFyZSBzb21lIHBlcmZlY3QgY29sbGluZWFyaXRpZXMgaW4gdGhpcyBtb2RlbCAtPiBlbGltaW5hdGUgaXQgdmlhIEFJQy9CSUMKc3VtbWFyeShtb2RlbC4wKQpwbG90KG1vZGVsLjApCgojQUlDCiNtb2RlbC5BSUM9c3RlcChtb2RlbC4wLCBrPTIpCiNzdW1tYXJ5KG1vZGVsLkFJQykKI3Bsb3QobW9kZWwuQUlDKQoKI0JJQwojbW9kZWwuQklDPXN0ZXAobW9kZWwuMCwgaz1sb2cobnJvdyhkYXRhX3RyYWluKSkpCgoKbW9kZWwuQklDPWxtKGZvcm11bGEgPSBsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhID0gZGF0YV90cmFpbikKc3VtbWFyeShtb2RlbC5CSUMpCnBsb3QobW9kZWwuQklDKQoKbW9kZWxpbnRlcmFjdCA9IGxtKGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgQ29uZGl0aW9uLjEgKyBPdmVyYWxsLlF1YWw6QmF0aHM6TmVpZ2hib3Job29kICsgT3ZlcmFsbC5RdWFsOkJhdGhzICsgT3ZlcmFsbC5RdWFsOk5laWdoYm9yaG9vZCArIE92ZXJhbGwuUXVhbCArIEJhdGhzICsgTmVpZ2hib3Job29kICsgT3ZlcmFsbC5RdWFsOkJhdGhzOmxvZyhUb3RhbC5Cc210LlNGKSArIE92ZXJhbGwuUXVhbDpsb2coVG90YWwuQnNtdC5TRikgKyBPdmVyYWxsLlF1YWw6R2FyYWdlLkNhcnM6bG9nKFRvdGFsLkJzbXQuU0YpOmxvZyhUb3RhbFNxKSArIE92ZXJhbGwuUXVhbDpHYXJhZ2UuQ2Fyczpsb2coVG90YWxTcSkgKyBPdmVyYWxsLlF1YWw6R2FyYWdlLkNhcnMgKyBPdmVyYWxsLlF1YWw6bG9nKFRvdGFsU3EpICsgR2FyYWdlLkNhcnM6bG9nKFRvdGFsLkJzbXQuU0YpOmxvZyhUb3RhbFNxKSArIEdhcmFnZS5DYXJzOmxvZyhUb3RhbFNxKSArIGxvZyhUb3RhbC5Cc210LlNGKTpsb2coVG90YWxTcSkgKyBHYXJhZ2UuQ2FycyArIGxvZyhUb3RhbC5Cc210LlNGKSArIGxvZyhUb3RhbFNxKSArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuRXhwb3N1cmUgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSwgZGF0YSA9IGRhdGFfdHJhaW4pCgptb2RlbC5pbnRlcmFjdC5yZWR1Y2VkPWxtKGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgQ29uZGl0aW9uLjEgKyBPdmVyYWxsLlF1YWwgKyAKICAgIEJhdGhzICsgTmVpZ2hib3Job29kICsgR2FyYWdlLkNhcnMgKyBsb2coVG90YWwuQnNtdC5TRikgKyAKICAgIGxvZyhUb3RhbFNxKSArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgQnNtdC5FeHBvc3VyZSArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEJlZHJvb20uQWJ2R3IgKyAKICAgIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgRmlyZXBsYWNlcyArIAogICAgUGF2ZWQuRHJpdmUgKyBPdmVyYWxsLlF1YWw6R2FyYWdlLkNhcnMgKyBPdmVyYWxsLlF1YWw6bG9nKFRvdGFsU3EpICsgCiAgICBHYXJhZ2UuQ2Fyczpsb2coVG90YWxTcSkgKyBPdmVyYWxsLlF1YWw6R2FyYWdlLkNhcnM6bG9nKFRvdGFsU3EpLCAKICAgIGRhdGEgPSBkYXRhX3RyYWluKQpzdW1tYXJ5KG1vZGVsLmludGVyYWN0LnJlZHVjZWQpCnBsb3QobW9kZWwuaW50ZXJhY3QucmVkdWNlZCkKCgpzdGVwKG1vZGVsaW50ZXJhY3QsIGs9bG9nKG5yb3coZGF0YV90cmFpbikpKQpsaWJyYXJ5KHRyZWUpCnRyZWUgPSB0cmVlKGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgICAgICAgICAgICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgQnNtdC5RdWFsICsgCiAgICAgICAgICAgICAgIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgQmF0aHMgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyAKICAgICAgICAgICAgICAgRmlyZXBsYWNlcyArIEdhcmFnZS5DYXJzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIAogICAgICAgICAgICAgICBsb2coVG90YWxTcSksIGRhdGEgPSBkYXRhX3RyYWluKQojIHVzZSBjcm9zcy12YWxpZGF0aW9uIHRvIHBydW5lCmN2LnRyZWUgPSBjdi50cmVlKHRyZWUpCnBsb3QoY3YudHJlZSRzaXplLCBjdi50cmVlJGRldiwgdHlwZT0nYicpCmJiID0gY3YudHJlZSRzaXplW3doaWNoLm1pbihjdi50cmVlJGRldildCgojIHBydW5lIHRoZSB0cmVlIGZvciB0aGUgc2l6ZSB0aGF0IGlzIG9wdGltYWwgYmFzZWQgb24gQ1YKcHJ1bmUudHJlZT1wcnVuZS50cmVlKHRyZWUsIGJlc3Q9YmIpCgojIHBsb3QgdGhlIHBydW5lZCB0cmVlCnBsb3QocHJ1bmUudHJlZSkKdGV4dChwcnVuZS50cmVlLCBwcmV0dHk9MCkKcHJ1bmUudHJlZQoKc3VtbWFyeShtb2RlbC5CSUMpCnBsb3QobW9kZWwuQklDKQoKIyBFeHBsb3JpbmcgdGhlIHJlbWFpbmluZyBwcmVkaWN0b3JzIHJlbGF0aW9uc2hpcCB0byBwcmljZQojIHBsb3QobG9nKGRhdGFfdHJhaW4kTG90LkFyZWEpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiROZWlnaGJvcmhvb2QpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRDb25kaXRpb24uMSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJE92ZXJhbGwuUXVhbCksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJE92ZXJhbGwuQ29uZCksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEhvdXNlQWdlKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kQnNtdC5RdWFsKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kQnNtdC5FeHBvc3VyZSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KGxvZyhkYXRhX3RyYWluJFRvdGFsLkJzbXQuU0YpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRIZWF0aW5nLlFDKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kQ2VudHJhbC5BaXIpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRCYXRocyksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEJlZHJvb20uQWJ2R3IpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRLaXRjaGVuLkFidkdyKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kS2l0Y2hlbi5RdWFsKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kRnVuY3Rpb25hbCksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEZpcmVwbGFjZXMpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRQYXZlZC5Ecml2ZSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEdhcmFnZS5DYXJzKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kR2FyYWdlLkNhcnMpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdChsb2coMStkYXRhX3RyYWluJFBvb2wuQXJlYSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KGxvZyhkYXRhX3RyYWluJFRvdGFsU3EpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCgp0ZXJtcGxvdChtb2RlbC5CSUMsIHBhcnRpYWwucmVzaWQgPSBUUlVFLCBjb2wucmVzID0gInB1cnBsZSIsIGNleCA9IDAuNSwKICAgICAgICAgcnVnID0gVCwgc2UgPSBULCBzbW9vdGggPSBwYW5lbC5zbW9vdGgpCgoKCiMgVGhlcmUgYXJlIDMgaGlnaCBsZXZlcmFnZSBwb2ludHMgLSBtYXkgd2FudCB0byBleGNsdWRlIHRoZW0KaGg8LWhhdHZhbHVlcyhtb2RlbC5CSUMpCmlkPC13aGljaChoaD09MSkKCnBsb3QoaGF0dmFsdWVzKG1vZGVsLkJJQyksIHR5cGUgPSAiaCIpCgoKYGBgCgoKCgoKCgpgYGB7cn0KCiMjIFByZXBhcmUgdGVzdCBkYXRhCmxvYWQoImFtZXNfdGVzdC5SZGF0YSIpCgpkYXRhPWFtZXNfdGVzdAoKZGF0YSA8LSBkYXRhICAlPiUgIAogICNmaWx0ZXIoIWlzLm5hKExvdC5Gcm9udGFnZSkpICU+JSAKICBtdXRhdGUoTVMuU3ViQ2xhc3M9IGZhY3RvcihNUy5TdWJDbGFzcykpICU+JQogIG11dGF0ZShBbGxleSA9ICBmYWN0b3IoQWxsZXksIGxldmVscyA9IGxldmVscyhhZGROQShBbGxleSkpLCBsYWJlbHMgPSBjKGxldmVscyhBbGxleSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEhvdXNlQWdlID0gWXIuU29sZC0gcG1heChZZWFyLkJ1aWx0LCBZZWFyLlJlbW9kLkFkZCkpICAlPiUKICAjZmlsdGVyKCFpcy5uYShNYXMuVm5yLkFyZWEpKSAgJT4lIAogIG11dGF0ZShCc210LllOID0gMSooIWlzLm5hKEJzbXQuUXVhbCkpKSAlPiUKICBtdXRhdGUoQnNtdC5RdWFsID0gIGZhY3RvcihCc210LlF1YWwsIGxldmVscyA9IGxldmVscyhhZGROQShCc210LlF1YWwpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdC5RdWFsKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdC5RdWFsID0gcmVsZXZlbChCc210LlF1YWwsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoQnNtdC5Db25kID0gIGZhY3RvcihCc210LkNvbmQsIGxldmVscyA9IGxldmVscyhhZGROQShCc210LkNvbmQpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdC5Db25kKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdC5Db25kID0gcmVsZXZlbChCc210LkNvbmQsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9ICBmYWN0b3IoQnNtdC5FeHBvc3VyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuRXhwb3N1cmUpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdC5FeHBvc3VyZSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXQuRXhwb3N1cmUgPSByZWxldmVsKEJzbXQuRXhwb3N1cmUsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjE9ICBmYWN0b3IoQnNtdEZpbi5UeXBlLjEsIGxldmVscyA9IGxldmVscyhhZGROQShCc210RmluLlR5cGUuMSkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210RmluLlR5cGUuMSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4xID0gcmVsZXZlbChCc210RmluLlR5cGUuMSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMj0gIGZhY3RvcihCc210RmluLlR5cGUuMiwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4yKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4yKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjIgPSByZWxldmVsKEJzbXRGaW4uVHlwZS4yLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKFgxMi5TRj0gWDFzdC5GbHIuU0YrIFgybmQuRmxyLlNGKSAgJT4lICAgCiAgI2ZpbHRlcighaXMubmEoQnNtdC5GdWxsLkJhdGgpKSAlPiUKICAjZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IDEqKEZpcmVwbGFjZXM+MCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSAgZmFjdG9yKEZpcmVwbGFjZS5RdSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEZpcmVwbGFjZS5RdSkpLCBsYWJlbHMgPSBjKGxldmVscyhGaXJlcGxhY2UuUXUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSByZWxldmVsKEZpcmVwbGFjZS5RdSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuWU4gPSAxKighaXMubmEoR2FyYWdlLkNvbmQpKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlR5cGUgPSByZWxldmVsKEdhcmFnZS5UeXBlLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5GaW5pc2ggPSAgZmFjdG9yKEdhcmFnZS5GaW5pc2gsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuRmluaXNoKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5GaW5pc2gpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShHYXJhZ2UuRmluaXNoID0gcmVsZXZlbChHYXJhZ2UuRmluaXNoLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5RdWFsID0gIGZhY3RvcihHYXJhZ2UuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5RdWFsKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5RdWFsKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlF1YWwgPSByZWxldmVsKEdhcmFnZS5RdWFsLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5Db25kID0gIGZhY3RvcihHYXJhZ2UuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5Db25kKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkNvbmQgPSByZWxldmVsKEdhcmFnZS5Db25kLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKFBvcmNoLkFyZWEgPSBXb29kLkRlY2suU0YrIE9wZW4uUG9yY2guU0YrRW5jbG9zZWQuUG9yY2grWDNTc24uUG9yY2ggKyBTY3JlZW4uUG9yY2gpICU+JQogIG11dGF0ZShQb29sLllOID0gMSooUG9vbC5BcmVhPjApKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKFBvb2wuUUMgPSByZWxldmVsKFBvb2wuUUMsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICBtdXRhdGUoWXIuU29sZCA9IGFzLmZhY3RvcihZci5Tb2xkKSkgJT4lCiAgZHBseXI6OnNlbGVjdCgtR2FyYWdlLllyLkJsdCkgJT4lCiAgbXV0YXRlKENvbmRpdGlvbi4xID0gYXMuY2hhcmFjdGVyKENvbmRpdGlvbi4xKSkgJT4lCiAgbXV0YXRlKEtpdGNoZW4uUXVhbD1wbHlyOjptYXB2YWx1ZXMoS2l0Y2hlbi5RdWFsLCBmcm9tID0gYygiUG8iLCAiRmEiLCAiVEEiLCJHZCIsICJFeCIgKSwgdG8gPSBjKCIxIiwgIjIiLCAiMyIsICI0IiwgIjUiKSkpICU+JQogIG11dGF0ZShLaXRjaGVuLlF1YWwgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihLaXRjaGVuLlF1YWwpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUM9cGx5cjo6bWFwdmFsdWVzKEhlYXRpbmcuUUMsIGZyb20gPSBjKCJQbyIsICJGYSIsICJUQSIsIkdkIiwgIkV4IiApLCB0byA9IGMoIjEiLCAiMiIsICIzIiwgIjQiLCAiNSIpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUMgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihIZWF0aW5nLlFDKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSBkcm9wbGV2ZWxzKEJzbXQuUXVhbCkpICU+JQogIG11dGF0ZShGdW5jdGlvbmFsID0gZHJvcGxldmVscyhGdW5jdGlvbmFsKSkgJT4lCiAgbXV0YXRlKFJvb2YuTWF0bCA9IGRyb3BsZXZlbHMoUm9vZi5NYXRsKSkKCgoKCmluZF9yYWlsPC13aGljaChkYXRhJENvbmRpdGlvbi4xPT0iUlJObiIgfCBkYXRhJENvbmRpdGlvbi4xPT0iUlJBbiIgfCBkYXRhJENvbmRpdGlvbi4xPT0iUlJOZSIgfCBkYXRhJENvbmRpdGlvbi4xPT0iUlJBZSIpCmluZF9wYXJrPC13aGljaChkYXRhJENvbmRpdGlvbi4xPT0iUG9zTiIgfCBkYXRhJENvbmRpdGlvbi4xPT0iUG9zQSIpCmRhdGEkQ29uZGl0aW9uLjFbaW5kX3JhaWxdPC0iUmFpbCIKZGF0YSRDb25kaXRpb24uMVtpbmRfcGFya108LSJQYXJrIgoKCgpkYXRhID0gZGF0YSAlPiUgCiAgbXV0YXRlKENvbmRpdGlvbi4xID0gZmFjdG9yKENvbmRpdGlvbi4xKSkgJT4lCiAgbXV0YXRlKENvbmRpdGlvbi4xID0gcmVsZXZlbChDb25kaXRpb24uMSwgcmVmPSJOb3JtIikpIAoKZGF0YV90ZXN0PWRhdGEKCgpkYXRhX3Rlc3QkQnNtdC5FeHBvc3VyZVt3aGljaChkYXRhX3Rlc3QkQnNtdC5FeHBvc3VyZT09IiIpXTwtIk5vbmUiCmRhdGFfdGVzdCRCc210LkV4cG9zdXJlPC1kcm9wbGV2ZWxzKGRhdGFfdGVzdCRCc210LkV4cG9zdXJlKQoKCgpkYXRhX3Rlc3QkUG9vbC5BcmVhPC1kYXRhX3Rlc3QkUG9vbC5BcmVhKzEKZGF0YV90ZXN0JFRvdGFsLkJzbXQuU0Y8LWRhdGFfdGVzdCRUb3RhbC5Cc210LlNGKzEKCgojIGV4dHJhY3QgdGhlIHRydXRoClkgPSBkYXRhX3Rlc3QkcHJpY2UKCiMgbWFrZSBwcmVkaWN0aW9uIGJhc2VkIG9uIHNwZWNpZmljIG1vZGVsCgoKClloYXQgPSBwcmVkaWN0KG1vZGVsLCBuZXdkYXRhPWRhdGFfdGVzdCwgaW50ZXJ2YWw9InByZWRpY3QiKQoKIyBkZXBlbmRpbmcgb24gdGhlIHJlc3BvbnNlIHRyYW5zZm9ybWF0aW9uClloYXQgPSBleHAoWWhhdCkKCiNGb3IgY2FsY3VsYXRpbmcgVG9wIDEwIHVuZGVydmFsdWVkIGFuZCBvdmVydmFsdWVkIGhvbWVzCm92ZXIudW5kZXIgPSBkYXRhLmZyYW1lKHZhbHVlLmRpZmYgPSBZaGF0L1kpCnVuZGVydmFsdWVkPWRwbHlyOjp0b3BfbihvdmVyLnVuZGVyLCAxMCwgdmFsdWUuZGlmZikKb3ZlcnZhbHVlZD1kcGx5cjo6dG9wX24ob3Zlci51bmRlciwgMTAsIGRlc2ModmFsdWUuZGlmZikpCnRvcC4xMCA9IGRhdGEuZnJhbWUodW5kZXIgPSBkcGx5cjo6YXJyYW5nZSh1bmRlcnZhbHVlZCwgZGVzYyh2YWx1ZS5kaWZmKSksIG92ZXIgPSBkcGx5cjo6YXJyYW5nZShvdmVydmFsdWVkLHZhbHVlLmRpZmYpKQpsaWJyYXJ5KGtuaXRyKQprYWJsZSh0b3AuMTAsIGNvbC5uYW1lcz1jKCJUb3AgMTAgVW5kZXJ2YWx1ZWQgSG91c2VzIiwgIlRvcCAxMCBPdmVydmFsdWVkIEhvdXNlcyIpKQoKCiMgbmFtZSBkYXRhZnJhbWUgYXMgcHJlZGljdGlvbnMhIERPIE5PVCBDSEFOR0UKcHJlZGljdGlvbnMgPSBhcy5kYXRhLmZyYW1lKFloYXQpCnByZWRpY3Rpb25zJFBJRCA9IGRhdGFfdGVzdCRQSUQKCnNhdmUocHJlZGljdGlvbnMsIGZpbGU9InByZWRpY3QuUmRhdGEiKQoKcGVyZm9ybWFuY2UoWSwgWWhhdCkKCm1lYW4oWS1ZaGF0KQogIAptYXgoYWJzKFktWWhhdCkpCiAgCm1lYW4oYWJzKFktWWhhdFsxXSkpCiAgCnNxcnQobWVhbigoWS1ZaGF0KV4yKSkKYGBgCgoKYGBge3J9CiMgTGFzc28gZm9yIHRoZSBzaW1wbGlmaWVkIG1vZGVsIChmcm9tIEJJQykKbGlicmFyeShnbG1uZXQpCgpYLnRyYWluID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgICAgICAgICAgICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgQnNtdC5RdWFsICsgCiAgICAgICAgICAgICAgIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgQmF0aHMgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyAKICAgICAgICAgICAgICAgRmlyZXBsYWNlcyArIEdhcmFnZS5DYXJzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIAogICAgICAgICAgICAgICBsb2coVG90YWxTcSksIGRhdGEgPSBkYXRhX3RyYWluKQoKClgudGVzdCA9IG1vZGVsLm1hdHJpeChsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhID0gZGF0YV90ZXN0KQoKCm1vZGVsLmxhc3NvID0gZ2xtbmV0KFgudHJhaW4sIGxvZyhkYXRhX3RyYWluJHByaWNlKSwgYWxwaGE9MSkKY3YubGFzc28gPSBjdi5nbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0xKQp5aGF0Lmxhc3NvID0gZXhwKHByZWRpY3QobW9kZWwubGFzc28sIHM9Y3YubGFzc28kbGFtYmRhLm1pbiwgdHlwZT0icmVzcG9uc2UiLCBuZXd4ID0gWC50ZXN0KSkKCnNxcnQobWVhbigoKHloYXQubGFzc28pLWRhdGFfdGVzdCRwcmljZSleMikpCgoKIyBSaWRnZSBmb3IgdGhlIHNpbXBsaWZpZWQgbW9kZWwgKGZyb20gQklDKQoKbW9kZWwucmlkZ2UgPSBnbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0wKQpjdi5yaWRnZSA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YV90cmFpbiRwcmljZSksIGFscGhhPTApCnloYXQucmlkZ2UgPSBwcmVkaWN0KG1vZGVsLnJpZGdlLCBzPWN2LnJpZGdlJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCnNxcnQobWVhbigoZXhwKHloYXQucmlkZ2UpLWRhdGFfdGVzdCRwcmljZSleMikpCgoKCgoKYGBgCgoKCgpgYGB7cn0KCiMgQk1BIGZvciBzaW1wbGUgbW9kZWwgKGZyb20gQklDKQoKbGlicmFyeShCQVMpCm1vZGVsLmJtYSA9IGJhcy5sbShsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhPWRhdGFfdHJhaW4sICBwcmlvcj0iZy1wcmlvciIsIGE9bnJvdyhkYXRhX3RyYWluKSwgbW9kZWxwcmlvcj11bmlmb3JtKCksICBtZXRob2Q9Ik1DTUMiLCBNQ01DLml0ZXJhdGlvbnMgPSAyMDAwMDAsIHRoaW4gPSAyMCkKCgoKCnlfcHJlZDwtcHJlZGljdC5iYXMobW9kZWwuYm1hLG5ld2RhdGE9ZGF0YV90ZXN0LCBlc3RpbWF0b3I9IkJQTSIsIHByZWRpY3Rpb249VFJVRSwgc2UuZml0PVRSVUUpCgp5X2hhdDwtZXhwKGNvbmZpbnQucHJlZC5iYXMoeV9wcmVkKSkKCnlfaGF0PSB5X2hhdFssYygzLDEsMildCgpwZXJmb3JtYW5jZShkYXRhX3Rlc3QkcHJpY2UsIHlfaGF0KQoKYGBgCmBgYHtyfQojRXhwZXJpbWVudGF0aW9uIHdpdGggR0FNCmxpYnJhcnkobWdjdikKc2V0LnNlZWQoMSkKCiMjR0FNIG1vZGVsIDE6IHdpdGhvdXQgaW50ZXJhY3Rpb24KCmdhbV9tb2RlbF8xID0gYmFtKHByaWNlIH4gcyhMb3QuQXJlYSwgaz0zKSArIHMoTmVpZ2hib3Job29kLCBrPTMsIGJzPSJyZSIpICsgcyhDb25kaXRpb24uMSwgaz0zLCBicz0icmUiKSArIAogICAgICAgICAgICAgICBPdmVyYWxsLlF1YWwgKyBPdmVyYWxsLkNvbmQgKyBzKEhvdXNlQWdlLCBrPTMpICsgcyhGb3VuZGF0aW9uLCBrPTMsIGJzPSJyZSIpICsgcyhCc210LlF1YWwsIGs9MywgYnM9InJlIikgKyAKICAgICAgICAgICAgICAgcyhCc210LkV4cG9zdXJlLCBrPTMsIGJzPSJyZSIpICsgcyhUb3RhbC5Cc210LlNGLCBrPTMpICsgcyhIZWF0aW5nLlFDLCBrPTMsIGJzPSJyZSIpICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgcyhCYXRocywgaz0zKSArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgcyhLaXRjaGVuLlF1YWwsIGs9MywgYnM9InJlIikgKyBzKEZ1bmN0aW9uYWwsIGs9MywgYnM9InJlIikgKwogICAgICAgICAgICAgICBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIHMoUG9vbC5BcmVhLCBrPTMpICsgCiAgICAgICAgICAgICAgIHMoVG90YWxTcSwgaz0zKSwgZmFtaWx5PWdhdXNzaWFuKGxpbmsgPSAiaWRlbnRpdHkiKSwgZGF0YSA9IGRhdGFfdHJhaW4pCgpZaGF0ID0gcHJlZGljdC5iYW0oZ2FtX21vZGVsXzEsIG5ld2RhdGEgPSBkYXRhX3Rlc3QsIHR5cGUgPSAicmVzcG9uc2UiKQpgYGAKCgpgYGB7cn0KbGlicmFyeShnZ3Bsb3QyKQojRGF0YSBWaXplcwpuZWlnaGJvcmhvb2QucHJpY2UucmFuZ2UgPSBnZ3Bsb3QoZGF0YV90cmFpbiwgYWVzKHggPSByZW9yZGVyKE5laWdoYm9yaG9vZCwgZGVzYyhwcmljZSksIG1lZGlhbiksIHkgPSBwcmljZSwgZmlsbD1wcmljZSkpICsgZ2VvbV9ib3hwbG90KGNvbG91cj0iZGFya3NsYXRlYmx1ZSIsZmlsbD10ZXJyYWluLmNvbG9ycygyOCwgYWxwaGE9MSkpICsgdGhlbWVfbGlnaHQoKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT00NSxoanVzdD0xLHZqdXN0PTEpKSArIGxhYnMoeCA9ICJOZWlnaGJvcmhvb2QiLCB5PSJQcmljZSIsIHRpdGxlPSJCb3hwbG90IG9mIFByaWNlIGJ5IE5laWdoYm9yaG9vZCIpICsgdGhlbWUocGxvdC50aXRsZSA9IGVsZW1lbnRfdGV4dChoanVzdD0wLjUpKQoKcGxvdChuZWlnaGJvcmhvb2QucHJpY2UucmFuZ2UpCgpsaWJyYXJ5KHRyZWVtYXApCgp0cmVlLmRhdGEgPSBkYXRhX3RyYWluICU+JSBtdXRhdGUocHJpY2UuYmluID0gaWZlbHNlKHByaWNlIDw9IDUwMDAwLCAiPD0gJDUwLDAwMCIsIGlmZWxzZShwcmljZSA+IDUwMDAwICYgcHJpY2UgPD0gMTAwMDAwLCAiJDUwLDAwMC0xMDAsMDAwIiwgaWZlbHNlKHByaWNlID4gMTAwMDAwICYgcHJpY2UgPD0gMjAwMDAwLCAiJDEwMCwwMDAtMjAwLDAwMCIsIGlmZWxzZShwcmljZSA+IDIwMDAwMCAmIHByaWNlIDw9IDMwMDAwMCwgIiQyMDAsMDAwLTMwMCwwMDAiLCBpZmVsc2UocHJpY2UgPiAzMDAwMDAgJiBwcmljZSA8PSA0MDAwMDAsICIkMzAwLDAwMC00MDAsMDAwIiwgaWZlbHNlKHByaWNlID4gNDAwMDAwICYgcHJpY2UgPD0gNTAwMDAwLCAiJDQwMCwwMDAtNTAwLDAwMCIsICI+ICQ1MDAsMDAwIikpKSkpKSkKdHJlZS5kYXRhID0gYXMuZGF0YS5mcmFtZSh0cmVlLmRhdGEpCgphcmVhLnBsb3QgPSB0cmVlbWFwKHRyZWUuZGF0YSwgaW5kZXg9YygicHJpY2UuYmluIiwgInByaWNlIiksIHZTaXplPSJMb3QuQXJlYSIsIHR5cGU9ImluZGV4IiwgcGFsZXR0ZT0iWWxHbiIsIGFsZ29yaXRobT0icGl2b3RTaXplIiwgdGl0bGU9IkVmZmVjdCBvZiBMb3QgQXJlYSBvbiBQcmljZSBSYW5nZSIsIGZvbnRzaXplLnRpdGxlID0gMTQpCgphcmVhLnBsb3QgPSB0cmVlbWFwKHRyZWUuZGF0YSwgaW5kZXg9YygicHJpY2UuYmluIiwgInByaWNlIiksIHZTaXplPSJUb3RhbFNxIiwgdHlwZT0iaW5kZXgiLCBwYWxldHRlPSJPcmFuZ2VzIiwgYWxnb3JpdGhtPSJwaXZvdFNpemUiLCB0aXRsZT0iRWZmZWN0IG9mIFRvdGFsIFNxdWFyZSBGb290YWdlIG9uIFByaWNlIFJhbmdlIiwgZm9udHNpemUudGl0bGUgPSAxNCkKCgpkYXRhX3RyYWluJE92ZXJhbGwuUXVhbCA9IGFzLm51bWVyaWMoZGF0YV90cmFpbiRPdmVyYWxsLlF1YWwpCnF1YWxpdHkucGxvdCA9IGdncGxvdChkYXRhX3RyYWluLCBhZXMoeCA9IE92ZXJhbGwuUXVhbCwgeT1wcmljZSkpICsgZ2VvbV9zbW9vdGgoY29sb3I9ImdvbGQiKSArIGdlb21fY291bnQoY29sb3I9InJlZDMiKSArIHRoZW1lX2J3KCkgKyBsYWJzKHg9Ik92ZXJhbGwgUXVhbGl0eSIsIHk9IlByaWNlIiwgdGl0bGU9IlByaWNlIGJ5IE92ZXJhbGwgUXVhbGl0eSIpICsgdGhlbWUocGxvdC50aXRsZSA9IGVsZW1lbnRfdGV4dChoanVzdD0wLjUpKQpzdXBwcmVzc01lc3NhZ2VzKHBsb3QocXVhbGl0eS5wbG90KSkKYGBgCgoKCgoKCgoKCgoKCmBgYHtyfQpsaWJyYXJ5KE1BU1MpCgojRnVsbCBNb2RlbAptb2RlbD1sbShsb2cocHJpY2UpIH4gTVMuU3ViQ2xhc3MgKyBNUy5ab25pbmcgKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhMb3QuQXJlYSkgKyAgU3RyZWV0ICsgQWxsZXkgKyBMb3QuU2hhcGUgKyBMYW5kLkNvbnRvdXIgKyBMb3QuQ29uZmlnICsgTGFuZC5TbG9wZSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBSb29mLlN0eWxlICsgUm9vZi5NYXRsICsgRXh0ZXJpb3IuMXN0ICsgRXh0ZXJpb3IuMm5kICsgTWFzLlZuci5UeXBlICsgbG9nKDErTWFzLlZuci5BcmVhKSArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXQuWU46QnNtdC5RdWFsICsgQnNtdC5ZTjpCc210LkNvbmQgKyBCc210LllOOkJzbXQuRXhwb3N1cmUgKyBUb3RhbC5Cc210LlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEVsZWN0cmljYWwgKyBsb2coWDEyLlNGKSArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuWU46RmlyZXBsYWNlLlF1ICsgR2FyYWdlLllOOkdhcmFnZS5UeXBlICsgR2FyYWdlLllOOkdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5ZTjpHYXJhZ2UuQ29uZCArIEdhcmFnZS5ZTjpHYXJhZ2UuUXVhbCArIFBhdmVkLkRyaXZlICsgbG9nKDErUG9vbC5BcmVhKSArIFBvb2wuWU46UG9vbC5RQyArIEZlbmNlICsgTWlzYy5WYWwgKyBNby5Tb2xkICsgWXIuU29sZCArIFNhbGUuVHlwZSAgKyBUb3RhbFNxLCBkYXRhPXRyYWluKQoKc3VtbWFyeShtb2RlbCkKYm94Y294KG1vZGVsKQojQklDIG1vZGVsIHNlbGVjdGlvbgpzdGVwKG1vZGVsLCBrPWxvZyhucm93KHRyYWluKSkpCgojc3RlcChtb2RlbCwgaz0yKQojIE9MRAojIG1vZGVsPWxtKGxvZyhwcmljZSl+bXMuc3ViY2xhc3MgKyBNUy5ab25pbmcgKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhhcmVhKSArIFN0cmVldCArIEFsbGV5ICsgTG90LlNoYXBlICsgTGFuZC5Db250b3VyICsgTG90LkNvbmZpZyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgUm9vZi5TdHlsZSArIEV4dGVyaW9yLjFzdCArIEV4dGVyaW9yLjJuZCArIE1hcy5WbnIuVHlwZSArIGxvZyhNYXMuVm5yLkFyZWEgKyAxKSArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyBFbGVjdHJpY2FsICsgbG9nKFgxc3QuRmxyLlNGKSArIGxvZyhYMm5kLkZsci5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgVG90Um1zLkFidkdyZCArIFBhdmVkLkRyaXZlICsgbG9nKFdvb2QuRGVjay5TRiArIDEpICsgbG9nKE9wZW4uUG9yY2guU0YgKyAxKSArIGxvZyhFbmNsb3NlZC5Qb3JjaCArIDEpICsgbG9nKFgzU3NuLlBvcmNoICsgMSkgKyBsb2coU2NyZWVuLlBvcmNoICsgMSkgKyBsb2coTWlzYy5WYWwgKyAxKSArIE1vLlNvbGQgKyBZci5Tb2xkICsgU2FsZS5UeXBlICsgbG9nKFRvdGFsU3EpLCBkYXRhPXRyYWluKQojdmlmKG1vZGVsKQojc3VtbWFyeShtb2RlbCkKCiNNb2RlbCBTZWxlY3Rpb24gYnkgQUlDCiNzdGVwKG1vZGVsLCBrPTIpCiMjQUlDX21vZGVsID0gbG0oZm9ybXVsYSA9IGxvZyhwcmljZSkgfiBQSUQgKyBhcmVhICsgTVMuU3ViQ2xhc3MgKyBNUy5ab25pbmcgKyAKIyAgICBMb3QuQXJlYSArIExvdC5TaGFwZSArIFV0aWxpdGllcyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyAKIyAgICBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICAgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgUm9vZi5TdHlsZSArIFJvb2YuTWF0bCArIEV4dGVyaW9yLjFzdCArIAojICAgIEV4dGVyaW9yLjJuZCArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXRGaW4uU0YuMSArIAojICAgIEJzbXRGaW4uU0YuMiArIEJzbXQuVW5mLlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIAojICAgIEVsZWN0cmljYWwgKyBYMXN0LkZsci5TRiArIFgybmQuRmxyLlNGICsgQnNtdC5GdWxsLkJhdGggKyAKIyAgICBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgR2FyYWdlLkNhcnMgKyBHYXJhZ2UuQXJlYSArIFBhdmVkLkRyaXZlICsgCiMgICAgV29vZC5EZWNrLlNGICsgT3Blbi5Qb3JjaC5TRiArIEVuY2xvc2VkLlBvcmNoICsgU2NyZWVuLlBvcmNoICsgCiMgICAgTW8uU29sZCArIFlyLlNvbGQgKyBoYXMuZmVuY2UgKyBoYXMuZmlyZXBsYWNlLCBkYXRhID0gdHJhaW4pCiNzdW1tYXJ5KEFJQ19tb2RlbCkKCiNNb2RlbCBTZWxlY3Rpb24gYnkgQklDCiNCSUNfbW9kZWwgPSBsbShmb3JtdWxhID0gbG9nKHByaWNlKSB+IFBJRCArIGFyZWEgKyBNUy5ab25pbmcgKyBMb3QuQXJlYSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgQmxkZy5UeXBlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICBZZWFyLkJ1aWx0ICsgWWVhci5SZW1vZC5BZGQgKyBCc210RmluLlNGLjEgKyBCc210RmluLlNGLjIgKyAKIyAgICBCc210LlVuZi5TRiArIENlbnRyYWwuQWlyICsgQnNtdC5GdWxsLkJhdGggKyBLaXRjaGVuLlF1YWwgKyAKIyAgICBGdW5jdGlvbmFsICsgR2FyYWdlLkNhcnMgKyBHYXJhZ2UuQXJlYSArIE9wZW4uUG9yY2guU0YgKyAKIyAgICBFbmNsb3NlZC5Qb3JjaCArIFNjcmVlbi5Qb3JjaCArIGhhcy5maXJlcGxhY2UsIGRhdGEgPSB0cmFpbikKI3N1bW1hcnkoQklDX21vZGVsKQoKI1VzZSBCb29zdGluZyB0byBmaW5kIGltcG9ydGFudCB2YXJpYWJsZXMKIwojbGlicmFyeShnYm0pCiMKI2Jvb3N0ID0gZ2JtKGxvZyhwcmljZSkgfiBQSUQgKyBhcmVhICsgTVMuWm9uaW5nICsgTG90LkFyZWEgKyAKIyAgICBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIAojICAgIFllYXIuQnVpbHQgKyBZZWFyLlJlbW9kLkFkZCArIEJzbXRGaW4uU0YuMSArIEJzbXRGaW4uU0YuMiArIAojICAgIEJzbXQuVW5mLlNGICsgQ2VudHJhbC5BaXIgKyBCc210LkZ1bGwuQmF0aCArIEtpdGNoZW4uUXVhbCArIAojICAgIEZ1bmN0aW9uYWwgKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5BcmVhICsgT3Blbi5Qb3JjaC5TRiArIAojICAgIEVuY2xvc2VkLlBvcmNoICsgU2NyZWVuLlBvcmNoICsgaGFzLmZpcmVwbGFjZSwgZGF0YSA9IHRyYWluLCBkaXN0cmlidXRpb249ImdhdXNzaWFuIiwgbi50cmVlcz01MDAwLCAjaW50ZXJhY3Rpb24uZGVwdGggPSAxLCBzaHJpbmthZ2U9MC4wMSwgdmVyYm9zZSA9IEYpCgojc3VtbWFyeShib29zdCkKCiMjU3BhcnNlIE1vZGVsIChyZW1vdmVkIHZhcmlhYmxlcyB0byBicmluZyB0aGUgbnVtYmVyIGJlbG93IDIwKQojc3BhcnNlX21vZGVsID0gbG0obG9nKHByaWNlKSB+IGFyZWEgKyBNUy5ab25pbmcgKyBMb3QuQXJlYSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICAgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgQnNtdEZpbi5TRi4xICsgQnNtdEZpbi5TRi4yICsgCiMgICAgQnNtdC5VbmYuU0YgKyBDZW50cmFsLkFpciArIEJzbXQuRnVsbC5CYXRoICsgS2l0Y2hlbi5RdWFsICsgCiMgICAgRnVuY3Rpb25hbCArIEdhcmFnZS5DYXJzICsgR2FyYWdlLkFyZWEgKyBoYXMuZmlyZXBsYWNlLCBkYXRhID0gdHJhaW4pCiNzdW1tYXJ5KHNwYXJzZV9tb2RlbCkKI3Bsb3Qoc3BhcnNlX21vZGVsKQojdGVybXBsb3Qoc3BhcnNlX21vZGVsLCAKIyAgICBkYXRhID0gdHJhaW4pCgojI1RyeWluZyB0byBkZXRlY3Qgbm9uLWxpbmVhcml0eQojYXR0YWNoKHRyYWluKQojcGxvdChsb2coYXJlYSksIGxvZyhwcmljZSkpCiNwbG90KGxvZyhMb3QuQXJlYSksIGxvZyhwcmljZSkpCiNwbG90KGxvZyhPdmVyYWxsLlF1YWwpLCBsb2cocHJpY2UpKQojcGxvdChsb2coT3ZlcmFsbC5Db25kKSwgbG9nKHByaWNlKSkKI3Bsb3QobG9nKFllYXIuQnVpbHQpLCBsb2cocHJpY2UpKQojcGxvdChsb2coWWVhci5SZW1vZC5BZGQsIGxvZyhwcmljZSkpKQojcGxvdChCc210RmluLlNGLjEsIGxvZyhwcmljZSkpCiNwbG90KEJzbXRGaW4uU0YuMiwgbG9nKHByaWNlKSkKI3Bsb3QoQnNtdC5VbmYuU0YsIGxvZyhwcmljZSkpCiNwbG90KEJzbXQuRnVsbC5CYXRoLCBsb2cocHJpY2UpKQojcGxvdChHYXJhZ2UuQ2FycywgbG9nKHByaWNlKSkKI3Bsb3QoR2FyYWdlLkFyZWEsIGxvZyhwcmljZSkpCiMKIyNPTEQgU2ltcGxlIG1vZGVsCiNtb2RlbDEgPSBsbShsb2cocHJpY2UpIH4gbG9nKGFyZWEpICsgTVMuWm9uaW5nICsgbG9nKExvdC5BcmVhKSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgbG9nKE92ZXJhbGwuUXVhbCkgKyBsb2coT3ZlcmFsbC5Db25kKSArIAojICAgIGxvZyhZZWFyLkJ1aWx0KSArIGxvZyhZZWFyLlJlbW9kLkFkZCkgKyBCc210RmluLlNGLjEgKyBCc210RmluLlNGLjIgKyAKIyAgICBCc210LlVuZi5TRiArIENlbnRyYWwuQWlyICsgQnNtdC5GdWxsLkJhdGggKwojICAgIEZ1bmN0aW9uYWwgKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5BcmVhICsgaGFzLmZpcmVwbGFjZSwgZGF0YSA9IHRyYWluWy1jKDE2OCwgNDYxLCA3ODcpLF0pCiNzdW1tYXJ5KG1vZGVsMSkKI3Bsb3QobW9kZWwxKQoKI1NpbXBsZSBtb2RlbCAoYmFzZWQgb24gQklDIHNlbGVjdGlvbikKbW9kZWwxID0gbG0oZm9ybXVsYSA9IGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICBkYXRhID0gdHJhaW4pCgojUGxvdCBQcmljZSBSYW5nZXMgb2YgZWFjaCBuZWlnaGJvcmhvb2QKbGlicmFyeShnZ3Bsb3QyKQoKbmVpZ2hib3Job29kLnByaWNlLnJhbmdlID0gZ2dwbG90KHRyYWluLCBhZXMoeCA9IGxvZyhwcmljZSksIHkgPSByZW9yZGVyKE5laWdoYm9yaG9vZCwgZGVzYyhsb2cocHJpY2UpKSwgbWVkaWFuKSkpICsKICBnZW9tX3BvbHlnb24oY29sb3I9IlNsYXRlR3JheSIsc2l6ZSA9IDEsIGFscGhhPTEpICsgdGhlbWVfbGlnaHQoKSArIGNvb3JkX2ZsaXAoKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpICsgbGFicyh5PSJOZWlnaGJvcmhvb2QiKQoKcGxvdChuZWlnaGJvcmhvb2QucHJpY2UucmFuZ2UpCmBgYAoKCgojIyMgTW9kZWwgRXZhbHVhdGlvbiBvbiBUZXN0IERhdGEKQ3JlYXRlIHByZWRpY3RlZCB2YWx1ZXMgZm9yIHByaWNlIHVzaW5nIHlvdXIgbW9kZWwgdXNpbmcgdGhlIHRlc3RpbmcgZGF0YQoKYGBge3IgcmVhZC10ZXN0LWRhdGF9CmxvYWQoImFtZXNfdGVzdC5SZGF0YSIpCgojVmFyaWFibGVzIHdpdGggTkEncyBhbmQgdGhlaXIgcHJvcG9ydGlvbiBvZiBtaXNzaW5nIGRhdGEKbWlzcyA9IGFwcGx5KGlzLm5hKGFtZXNfdHJhaW4pLCAyLCBzdW0pCm1pc3NfcHJvcCA9IHJvdW5kKG1pc3NbbWlzcz4wXS9ucm93KGFtZXNfdHJhaW4pLCAzKQpwcmludChtaXNzX3Byb3ApIAp3aGljaChtaXNzX3Byb3A+MC41KSAjIGZvdXIgZmVhdHVyZXMgaGF2ZSBncmVhdGVyIHRoYW4gNTAlIG9mIGRhdGEgbWlzc2luZwoKCiNDcmVhdGVkIGJpbmFyeSB2YXJpYWJsZXMgZm9yIHdoZXRoZXIgb3Igbm90IGEgaG91c2UgYXMgYW4gYWxsZXksIHBvb2wsIGZlbmNlLCBtaXNjLiBmZWF0dXJlLCBmaXJlcGxhY2UsIGJhc2VtZW50IG9yIGdhcmFnZSwgYXMgSSB0aG91Z2h0IGl0IG1heSBiZSBtb3JlIG1lYW5pbmdmdWwtIFRvbQojTWF5IGNvbnNpZGVyIGFkZGluZyBMb3QuRnJvbnRhZ2UgYmFjayBpbiwgYXMgaXQgb25seSBoYWQgMjg2IG1pc3NpbmcKI0hhdmUgdG8gZGVjaWRlIGJldHdlZW4gZmlsdGVyaW5nIG91dCBOL0EncyBmb3IgZGlmZmVyZW50IEdhcmFnZSB2YXJpYWJsZXMgYW5kIG9ubHkgaW5jbHVkaW5nICdoYXMuZ2FyYWdlJwojSGF2ZSB0byBkZWNpZGUgYmV0d2VlbiBmaWx0ZXJpbmcgb3V0IE4vQSdzIGZvciBkaWZmZXJlbnQgQmFzZW1lbnQgdmFyaWFibGVzIGFuZCBvbmx5IGluY2x1ZGluZyAnaGFzLmJhc2VtZW50Jwp0ZXN0IDwtIGFtZXNfdGVzdCAgJT4lICAgZmlsdGVyKCFpcy5uYShMb3QuRnJvbnRhZ2UpKSAlPiUgCiAgbXV0YXRlKEFsbGV5ID0gIGZhY3RvcihBbGxleSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEFsbGV5KSksIGxhYmVscyA9IGMobGV2ZWxzKEFsbGV5KSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoSG91c2VBZ2UgPSBZci5Tb2xkLSBwbWF4KFllYXIuQnVpbHQsIFllYXIuUmVtb2QuQWRkKSkgICU+JQogIGZpbHRlcighaXMubmEoTWFzLlZuci5BcmVhKSkgICU+JSAKICBtdXRhdGUoQnNtdC5ZTiA9ICFpcy5uYShCc210LlF1YWwpKSAlPiUKICAgIG11dGF0ZShCc210LlF1YWwgPSAgZmFjdG9yKEJzbXQuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LlF1YWwpLCAiTm9CYSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogICAgbXV0YXRlKEJzbXQuQ29uZCA9ICBmYWN0b3IoQnNtdC5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuQ29uZCksICJOb0JhIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9ICBmYWN0b3IoQnNtdC5FeHBvc3VyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuRXhwb3N1cmUpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdC5FeHBvc3VyZSksICJOb0JhIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgIG11dGF0ZShCc210RmluLlR5cGUuMT0gIGZhY3RvcihCc210RmluLlR5cGUuMSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4xKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4xKSwgIk5vQmEiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICAgIG11dGF0ZShCc210RmluLlR5cGUuMj0gIGZhY3RvcihCc210RmluLlR5cGUuMiwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4yKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4yKSwgIk5vQmEiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoWDEyLlNGPSBYMXN0LkZsci5TRisgWDJuZC5GbHIuU0YpICAlPiUgICAKICBmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IEZpcmVwbGFjZXM+MCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5RdSA9ICBmYWN0b3IoRmlyZXBsYWNlLlF1LCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmlyZXBsYWNlLlF1KSksIGxhYmVscyA9IGMobGV2ZWxzKEZpcmVwbGFjZS5RdSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5ZTiA9ICFpcy5uYShHYXJhZ2UuQ29uZCkpICU+JQogICAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkZpbmlzaCA9ICBmYWN0b3IoR2FyYWdlLkZpbmlzaCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5GaW5pc2gpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLkZpbmlzaCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoR2FyYWdlLlF1YWwgPSAgZmFjdG9yKEdhcmFnZS5RdWFsLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLlF1YWwpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLlF1YWwpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogICAgICBtdXRhdGUoR2FyYWdlLkNvbmQgPSAgZmFjdG9yKEdhcmFnZS5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLkNvbmQpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLkNvbmQpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShQb3JjaC5BcmVhID0gV29vZC5EZWNrLlNGKyBPcGVuLlBvcmNoLlNGK0VuY2xvc2VkLlBvcmNoK1gzU3NuLlBvcmNoICsgU2NyZWVuLlBvcmNoKSAlPiUKICBtdXRhdGUoUG9vbC5ZTiA9IFBvb2wuQXJlYT4wKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICAgIG11dGF0ZShZci5Tb2xkID0gYXMuZmFjdG9yKFlyLlNvbGQpKSAlPiUKICBkcGx5cjo6c2VsZWN0KC1HYXJhZ2UuWXIuQmx0KSAlPiUgZmlsdGVyKENvbmRpdGlvbi4xICE9ICJSUk5lIikgJT4lIGZpbHRlcihLaXRjaGVuLlF1YWwgIT0gIlBvIikKICAgICAgICAgICAgICAgICAgCmBgYAoKYGBge3IgcHJlZGljdC1tb2RlbDEsIGVjaG89RkFMU0V9ClloYXQgPSBwcmVkaWN0KG1vZGVsMSwgbmV3ZGF0YT10ZXN0LCBpbnRlcnZhbD0icHJlZGljdGlvbiIpCmBgYAoKWW91IHNob3VsZCBzYXZlIHlvdXIgcHJlZGljdGlvbnMgaW4gYSBkYXRhZnJhbWUgd2l0aCBjb2x1bW5zIGZvciBgUElEYCAgKHByb3BlcnR5IGlkZW50aWZpZXIpLCBgZml0YCwgcHJlZGljdGVkIHZhbHVlcyBvbiB0aGUgdGVzdCBkYXRhLCBhbmQgd2hlcmUgcG9zc2libGUgYGx3cmAgYW5kIGB1cHJgLCBsb3dlciBhbmQgdXBwZXIgOTUlIGludGVydmFsIGVzdGltYXRlcyBmb3IgcHJlZGljdGluZyBgcHJpY2VgLiAKCmBgYHtyIGNyZWF0ZSB9ClkgPSB0ZXN0JHByaWNlClloYXQgPSBleHAoWWhhdCkKCiNCaWFzCm1lYW4oWWhhdFssMV0gLSBZKQoKI01heGltdW0gRGV2aWF0aW9uCm1heChhYnMoWSAtIFloYXRbLDFdKSkKCiNNZWFuIEFic29sdXRlIERldmlhdGlvbgptZWFuKGFicyhZIC0gWWhhdFssMV0pKQoKI1JNU0UKc3FydChtZWFuKChZIC0gWWhhdFssMV0pXjIpKQoKI0NvdmVyYWdlCm1lYW4oWWhhdFssImx3ciJdIDwgWSAmIFloYXRbLCJ1cHIiXSA+IFkpCgojIG5hbWUgZGF0YWZyYW1lIGFzIHByZWRpY3Rpb25zISBETyBOT1QgQ0hBTkdFCnByZWRpY3Rpb25zID0gYXMuZGF0YS5mcmFtZShZaGF0KQpwcmVkaWN0aW9ucyRQSUQgPSB0ZXN0JFBJRApzYXZlKHByZWRpY3Rpb25zLCBmaWxlPSJwcmVkaWN0LlJkYXRhIikKYGBgCgpZb3VyIG1vZGVscyB3aWxsIGJlIGV2YWx1YXRlZCBvbiB0aGUgZm9sbG93aW5nIGNyaXRlcmlhIG9uIHRoZSB0ZXN0IGRhdGE6IAoKKu+CtyBCaWFzOiAgQXZlcmFnZSAoWWhhdC1ZKSAgcG9zaXRpdmUgdmFsdWVzIGluZGljYXRlIHRoZSBtb2RlbCB0ZW5kcyB0byBvdmVyZXN0aW1hdGUgcHJpY2UgKG9uIGF2ZXJhZ2UpIHdoaWxlIG5lZ2F0aXZlIHZhbHVlcyBpbmRpY2F0ZSB0aGUgbW9kZWwgdGVuZHMgdG8gdW5kZXJlc3RpbWF0ZSBwcmljZS4KCirvgrcgTWF4aW11bSBEZXZpYXRpb246ICBNYXggfFktWWhhdHwgLSAgaWRlbnRpZmllcyB0aGUgd29yc3QgcHJlZGljdGlvbiAgbWFkZSBpbiB0aGUgdmFsaWRhdGlvbiBkYXRhIHNldC4KCirvgrcgTWVhbiBBYnNvbHV0ZSBEZXZpYXRpb246ICBBdmVyYWdlIHxZLVloYXR8IC0gdGhlIGF2ZXJhZ2UgZXJyb3IgKHJlZ2FyZGxlc3Mgb2Ygc2lnbikuCgoqIFJvb3TvgrcgTWVhbiBTcXVhcmUgRXJyb3I6IFNxcnQgQXZlcmFnZSAoWS1ZaGF0KV4yCgoqIENvdmVyYWdlOiAgQXZlcmFnZSggbHdyIDwgWSA8IHVwcikgCgpJbiBvcmRlciB0byBoYXZlIGEgcGFzc2luZyB3ZXJja2VyIGJhZGdlLCB5b3VyIGZpbGUgZm9yIHByZWRpY3Rpb25zIG5lZWRzIHRvIGJlIHRoZSBzYW1lIGxlbmd0aCBhcyB0aGUgdGVzdCBkYXRhLCB3aXRoIHRocmVlIGNvbHVtbnM6ICBmaXR0ZWQgdmFsdWVzLCBsb3dlciBDSSBhbmQgdXBwZXIgQ0kgdmFsdWVzIGluIHRoYXQgb3JkZXIgd2l0aCBuYW1lcywgZml0LCBsd3IsIGFuZCB1cHIgcmVzcGVjdGl2ZWx5LgoKWW91IHdpbGwgYmUgYWJsZSB0byBzZWUgeW91ciBzY29yZXMgb24gdGhlIHNjb3JlIGJvYXJkIChjb21pbmcgc29vbiEpLiAgVGhleSB3aWxsIGJlIGluaXRpYWxpemVkIGJ5IGEgcHJlZGN0aW9uIGJhc2VkIG9uIHRoZSBtZWFuIGluIHRoZSB0cmFpbmluZyBkYXRhLgoKX01vZGVsIENoZWNrXyAtIFRlc3QgeW91ciBwcmVkaWN0aW9uIG9uIHRoZSBmaXJzdCBvYnNlcnZhdGlvbiBpbiB0aGUgdHJhaW5pbmcgYW5kIHRlc3QgZGF0YSBzZXQgdG8gbWFrZSBzdXJlIHRoYXQgdGhlIG1vZGVsIGdpdmVzIGEgcmVhc29uYWJsZSBhbnN3ZXIgYW5kIGluY2x1ZGUgdGhpcyBpbiBhIHN1cHBsZW1lbnQgb2YgeW91ciByZXBvcnQuIFRoaXMgc2hvdWxkIGJlIGRvbmUgQlkgSEFORCB1c2luZyBhIGNhbGN1bGF0b3IgKHRoaXMgbWVhbnMgdXNlIHRoZSByYXcgZGF0YSBmcm9tIHRoZSBvcmlnaW5hbCBkYXRhc2V0IGFuZCBtYW51YWxseSBjYWxjdWxhdGUgYWxsIHRyYW5zZm9ybWF0aW9ucyBhbmQgaW50ZXJhY3Rpb25zIHdpdGggeW91ciBjYWxjdWxhdG9yKSEgTW9kZWxzIHRoYXQgZG8gbm90IGdpdmUgcmVhc29uYWJsZSBhbnN3ZXJzIHdpbGwgYmUgZ2l2ZW4gYSBtaW5pbXVtIDIgbGV0dGVyIGdyYWRlIHJlZHVjdGlvbi4gQWxzbyBiZSBjYXJlZnVsIGFzIHlvdSBjYW5ub3QgdXNlIGNlcnRhaW4gdHJhbnNmb3JtYXRpb25zIFtsb2cgb3IgaW52ZXJzZSB4XSBpZiBhIHZhcmlhYmxlIGhhcyB2YWx1ZXMgb2YgMC4KCgoKCiMjIyBQYXJ0IElJOiBDb21wbGV4IE1vZGVsCgpJbiB0aGlzIHBhcnQgeW91IG1heSBnbyBhbGwgb3V0IGZvciBjb25zdHJ1Y3RpbmcgYSBiZXN0IGZpdHRpbmcgbW9kZWwgZm9yIHByZWRpY3RpbmcgaG91c2luZyBwcmljZXMgdXNpbmcgbWV0aG9kcyB0aGF0IHdlIGhhdmUgY292ZXJlZCB0aGlzIHNlbWVzdGVyLiAgWW91IHNob3VsZCBmZWVsIGZyZWUgdG8gdG8gY3JlYXRlIGFueSBuZXcgdmFyaWFibGVzIChzdWNoIGFzIHF1YWRyYXRpYywgaW50ZXJhY3Rpb24sIG9yIGluZGljYXRvciB2YXJpYWJsZXMsIHNwbGluZXMsIGV0YykuIFRoZSB2YXJpYWJsZSBgVG90YWxTcSA9IFgxc3QuRmxyLlNGK1gybmQuRmxyLlNGYCB3YXMgYWRkZWQgdG8gdGhlIGRhdGFmcmFtZSAodGhhdCBkb2VzIG5vdCBpbmNsdWRlIGJhc2VtZW50IGFyZWEsIHNvIHlvdSBtYXkgaW1wcm92ZSBvbiB0aGlzLiBBIHJlbGF0aXZlIGdyYWRlIGlzIGFzc2lnbmVkIGJ5IGNvbXBhcmluZyB5b3VyIGZpdCBvbiB0aGUgdGVzdCBzZXQgdG8gdGhhdCBvZiB5b3VyIGZlbGxvdyBzdHVkZW50cyB3aXRoIGJvbnVzIHBvaW50cyBhd2FyZGVkIHRvIHRob3NlIHdobyBzdWJzdGFudGlhbGx5IGV4Y2VlZCB0aGVpciBmZWxsb3cgc3R1ZGVudHMgYW5kIHBvaW50IHJlZHVjdGlvbnMgb2NjdXJyaW5nIGZvciBtb2RlbHMgd2hpY2ggZml0IGV4Y2VlZGluZ2x5IHBvb3JseS4gIAoKYGBge3J9CmxpYnJhcnkoZ2xtbmV0KQojRGVzaWduIG1hdHJpY2VzClgudHJhaW4gPSBtb2RlbC5tYXRyaXgobG9nKHByaWNlKSB+IGxvZyhMb3QuQXJlYSkgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgVG90YWwuQnNtdC5TRiArIAogICAgQ2VudHJhbC5BaXIgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIAogICAgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgQnNtdC5ZTjpCc210LkV4cG9zdXJlLCAKICAgIGRhdGEgPSBkYXRhKQoKWC50ZXN0ID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICBkYXRhID0gdGVzdCkKCiNGaXQgbGFzc28gcmVncmVzc2lvbgpwcmljZS5sYXNzbyA9IGdsbW5ldChYLnRyYWluLCBsb2coZGF0YSRwcmljZSksIGFscGhhPTEpCgpjdi5sYXNzbyA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YSRwcmljZSksIGFscGhhPTEpCgojT2J0YWluIHByZWRpY3Rpb24gb24gdGVzdCBkYXRhCnloYXQubGFzc28udGVzdCA9IHByZWRpY3QoYXBwLmxhc3NvLCBzPWN2Lmxhc3NvJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCiNDb21wdXRlIFJNU0UgZm9yIHRlc3QgZGF0YQpybXNlLmxhc3NvLnRlc3QgPSBybXNlKHRlc3QkcHJpY2UsIGV4cCh5aGF0Lmxhc3NvLnRlc3QpKQpybXNlLnRhYiRsYXNzbzwtcm1zZS5sYXNzby50ZXN0CmBgYAoKVXBkYXRlIHlvdXIgcHJlZGljdGlvbnMgdXNpbmcgeW91ciBjb21wbGV4IG1vZGVsIHRvIHByb3ZpZGUgcG9pbnQgZXN0aW1hdGVzIGFuZCBDSS4KCmBgYHtyIHByZWRpY3QtbW9kZWwyLCBlY2hvPUZBTFNFfQojIHJlcGxhY2UgbW9kZWwxIHdpdGggbW9kZWwyCnByZWRpY3Rpb25zID0gYXMuZGF0YS5mcmFtZShwcmVkaWN0KG1vZGVsMSwgbmV3ZGF0YT1hbWVzX3Rlc3QsIGludGVydmFsID0gInByZWQiKSkKcHJlZGljdGlvbnMkUElEID0gYW1lc190ZXN0JFBJRApzYXZlKHByZWRpY3Rpb25zLCBmaWxlPSJwcmVkaWN0LlJkYXRhIikKYGBgCgpZb3UgbWF5IGl0ZXJhdGUgaGVyZSBhcyBtdWNoIGFzIHlvdSBsaWtlIGV4cGxvcmluZyBkaWZmZXJlbnQgbW9kZWxzIHVudGlsIHlvdSBhcmUgc2F0aXNmaWVkIHdpdGggeW91ciByZXN1bHRzLgoKCgoKIyMjIFBhcnQgSUlJOiBXcml0ZSBVcAoKT25jZSB5b3UgYXJlIHNhdGlzZmllZCB3aXRoIHlvdXIgbW9kZWwsIHByb3ZpZGUgYSB3cml0ZSB1cCBvZiB5b3VyIGRhdGEgYW5hbHlzaXMgcHJvamVjdCBpbiBhIG5ldyBSbWQgZmlsZS9wZGYgZmlsZTogYHdyaXRldXAuUm1kYCBieSBjb3B5aW5nIG92ZXIgc2FsaWVudCBwYXJ0cyBvZiB5b3VyIFIgbm90ZWJvb2suIFRoZSB3cml0dGVuIGFzc2lnbm1lbnQgY29uc2lzdHMgb2YgZml2ZSBwYXJ0czoKCjEuIEV4cGxvcmF0b3J5IGRhdGEgYW5hbHlzaXMgKDIwIHBvaW50cyk6IG11c3QgaW5jbHVkZSB0aHJlZSBjb3JyZWN0bHkgbGFiZWxlZCBncmFwaHMgYW5kIGFuIGV4cGxhbmF0aW9uIHRoYXQgaGlnaGxpZ2h0IHRoZSBtb3N0IGltcG9ydGFudCBmZWF0dXJlcyB0aGF0IHdlbnQgaW50byB5b3VyIG1vZGVsIGJ1aWxkaW5nLgoKCjIuIERldmVsb3BtZW50IGFuZCBhc3Nlc3NtZW50IG9mIGFuIGluaXRpYWwgbW9kZWwgZnJvbSBQYXJ0IEkgKDEwIHBvaW50cykKCiogSW5pdGlhbCBtb2RlbDogbXVzdCBpbmNsdWRlIGEgc3VtbWFyeSB0YWJsZSBhbmQgYW4gZXhwbGFuYXRpb24vZGlzY3Vzc2lvbiBmb3IgdmFyaWFibGUgc2VsZWN0aW9uLiAgSW50ZXJwcmV0YXRpb24gb2YgY29lZmZpY2llbnRzIGRlc2lyYWJsZSBmb3IgZnVsbCBwb2ludHMuCgoqIE1vZGVsIHNlbGVjdGlvbjogbXVzdCBpbmNsdWRlIGEgZGlzY3Vzc2lvbgoKKiBSZXNpZHVhbDogbXVzdCBpbmNsdWRlIGEgcmVzaWR1YWwgcGxvdCBhbmQgYSBkaXNjdXNzaW9uCgoqIFJNU0U6IG11c3QgaW5jbHVkZSBhbiBSTVNFIGFuZCBhbiBleHBsYW5hdGlvbiAgKG90aGVyIGNyaXRlcmlhIGRlc2lyYWJsZSkKCiogTW9kZWwgdGVzdGluZzogbXVzdCBpbmNsdWRlIGFuIGV4cGxhbmF0aW9uCgoKCjMuIERldmVsb3BtZW50IG9mIHRoZSBmaW5hbCBtb2RlbCAoMjAgcG9pbnRzKQoKKiBGaW5hbCBtb2RlbDogbXVzdCBpbmNsdWRlIGEgc3VtbWFyeSB0YWJsZQoKKiBWYXJpYWJsZXM6IG11c3QgaW5jbHVkZSBhbiBleHBsYW5hdGlvbgoKKiBWYXJpYWJsZSBzZWxlY3Rpb24vc2hyaW5rYWdlOiBtdXN0IHVzZSBhcHByb3ByaWF0ZSBtZXRob2QgYW5kIGluY2x1ZGUgYW4gZXhwbGFuYXRpb24KCgoKNC4gQXNzZXNzbWVudCBvZiB0aGUgZmluYWwgbW9kZWwgKDI1IHBvaW50cykKCiogUmVzaWR1YWw6IG11c3QgaW5jbHVkZSBhIHJlc2lkdWFsIHBsb3QgYW5kIGEgZGlzY3Vzc2lvbgoKKiBSTVNFOiBtdXN0IGluY2x1ZGUgYW4gUk1TRSBhbmQgYW4gZXhwbGFuYXRpb24gIChvdGhlciBjcml0ZXJpYSBkZXNpcmFibGUpCgoqIE1vZGVsIGV2YWx1YXRpb246IG11c3QgaW5jbHVkZSBhbiBldmFsdWF0aW9uIGRpc2N1c3Npb24KCiogTW9kZWwgdGVzdGluZyA6IG11c3QgaW5jbHVkZSBhIGRpc2N1c3Npb24KCiogTW9kZWwgcmVzdWx0OiBtdXN0IGluY2x1ZGUgYSBzZWxlY3Rpb24gb2YgdGhlIHRvcCAxMCB1bmRlcnZhbHVlZCBhbmQgb3ZlcnZhbHVlZCBob3VzZXMKCgoKNS4gQ29uY2x1c2lvbiAoMTAgcG9pbnRzKTogbXVzdCBpbmNsdWRlIGEgc3VtbWFyeSBvZiByZXN1bHRzIGFuZCBhIGRpc2N1c3Npb24gb2YgdGhpbmdzIGxlYXJuZWQKCgoKIyMjIFBhcnQgSVYKQ3JlYXRlIHByZWRpY3Rpb25zIGZvciB0aGUgdmFsaWRhdGlvbiBkYXRhIGZyb20geW91ciBmaW5hbCBtb2RlbCBhbmQgd3JpdGUgb3V0IHRvIGEgZmlsZSBgcHJlZGljdGlvbi12YWxpZGF0aW9uLlJkYXRhYApUaGlzIHNob3VsZCBoYXZlIHRoZSBzYW1lIGZvcm1hdCBhcyB0aGUgbW9kZWxzIGluIFBhcnQgSSBhbmQgSUkuCgoxMCBwb2ludHMKCgoKIyMjIENsYXNzIFByZXNlbnRhdGlvbnMKCkVhY2ggR3JvdXAgc2hvdWxkIHByZXBhcmUgNSBzbGlkZXMgaW4gdGhlaXIgR2l0aHViIHJlcG86ICAoc2F2ZSBhcyBzbGlkZXMucGRmKQoKKiBNb3N0IGludGVyZXN0aW5nIGdyYXBoaWMgIChhIHBpY3R1cmUgaXMgd29ydGggYSB0aG91c2FuZCB3b3JkcyBwcml6ZSEpICAKCiogQmVzdCBNb2RlbCAobW90aXZhdGlvbiwgaG93IHlvdSBmb3VuZCBpdCwgd2h5IHlvdSB0aGluayBpdCBpcyBiZXN0KQoKKiBCZXN0IEluc2lnaHRzIGludG8gcHJlZGljdGluZyBTYWxlcyBQcmljZS4KCiogMiBCZXN0IEhvdXNlcyB0byBwdXJjaGFzZSAgKGFuZCB3aHkpCgoqIEJlc3QgVGVhbSBOYW1lL0dyYXBoaWMKCldlIHdpbGwgc2VsZWN0IHdpbm5lcnMgYmFzZWQgb24gdGhlIGFib3ZlIGNyaXRlcmlhIGFuZCBvdmVyYWxsIHBlcmZvcm1hbmNlLgoKCkZpbmFsbHkgeW91ciByZXBvIHNob3VsZCBoYXZlOiBgd3JpdGV1cC5SbWRgLCBgd3JpdGV1cC5wZGZgLCBgc2xpZGVzLlJtZGAgKGFuZCB3aGF0ZXZlciBvdXRwdXQgeW91IHVzZSBmb3IgdGhlIHByZXNlbnRhdGlvbikgYW5kIGBwcmVkaWN0LlJkYXRhYCBhbmQgYHByZWRpY3QtdmFsaWRhdGlvbi5SZGF0YWAuCg==